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FOREWORD 


Air  Force  contract  No.  AF  04(695)-228,  under  Capt.  S.  Gavcus,  Jr.  as 
Project  Officer  with  Dr.  J.  Kaspar,  Aerospace  Corporation  as  Technical 
Advisor,  embodies  three  major  programs:  (1)  a  rigorous  theoretical  and 
experimental  information  extraction  study,  (2)  a  high  bit  density  lead 
selenide  array  fabrication  program,  and  (3)  an  experimental  demonstra¬ 
tion  of  data  preprocessing  capability  based  on  the  results  of  the 
extraction  study  and  state-of-the-art  fabrication  capability. 

Since  the  first  phase  of  the  information  extraction  study  is  of  more  general 
interest  than  the  technique  fabrication  program  it  appears  desirable  to 
separate  the  two  as  a  means  of  providing  wider  circulation  to  the  study 
effort  results.  For  this  reason  Vol.  II  of  this  Technical  Documentary 
Report  is  devoted  solely  to  a  detailed  review  of  the  information  extraction 
study  to  date.  Vol.  I  abstracts  salient  features  of  the  study  program 
as  they  apply  specifically  to  the  problem  at  hand,  as  well  as  detailing 
technique  development  accomplished  thus  far. 


H.  Graff 
Project  Manager 
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ABSTRACT 


The  reported  work  complements  present  detection  theory  by  providing  a 
useful  mathematical  statement  of  the  ideal  requirements  for  systems  en¬ 
deavoring  to  detect  time -variant  stochastic  signals  in  the  presence  of 
Gaussian  noise.  The  a  posteriori  probability  approach  developed  by 
P.  M.  Woodward  is  employed  to  state  the  sufficiency  conditions  in  general 
form;  these  are  then  reduced  to  a  useful  set  of  requirements  for  an  ideal 
detection  system  through  series  representation  of  arbitrary  signals. 

The  results  show  that  a  detection  system  can  only  provide  estimates  of 
relevant  information  appearing  in  a  signal-plus -noise  quantity;  inferences 
drawn  from  these  estimates  subsequent  to  detection  will  say  something 
about  signal  present.  The  results  clearly  state,  therefore,  that  a  detec¬ 
tion  system  is  fundamentally  an  information  detector,  and  not  a  detector 
of  signals. 

1  Present  detection  methods  are  shown  to  be  approximations  to  the  ideal 
requirements  developed.  The  ideal  requirements  are  evolved  in  a  form 
which  permit  possible  logical  improvements  to  present  methods. 

Subsequent  signal  processing  of  detection  system  output  receives  some 
attention.  Maximum  likelihood  is  considered,  as  well  as  an  approximate 
system  model  based  on  a  particular  interpretation  of  "psuedo-constant 
signal"  detection  which  leads  to  maximum  use  of  detected  quantity. 
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PREFACE 


This  report  summarizes  the  first  phase  of  a  study  program  embodied  in 
Air  Force  contract  No.  AF  04(695) -228,  Space  Systems  Division,  under 
Capt.  S.  Gavcus,  Jr.  as  Project  Officer  and  under  the  technical  direction 
of  Dr.  J.  Kaspar,  Aerospace  Corporation.  The  study  aspects  of  this 
contract  are  aimed  at  establishing  a  firm  mathematical  base  for  signal 
detection  from  which  requirements  for  passive  infrared  systems  can  be 
determined. 

The  basic  infrared  detection  problem  was  outlined  by  Dr.  Kaspar.  It  was 
at  his  request  that  the  work  of  P.  M.  Woodward  be  the  basis  for  the  study 
approach.  Credit  for  any  valuable  progress  in  the  reported  work,  there¬ 
fore,  must  largely  be  given  to  his  direction.  Any  shortcomings  in  this 
work  are  the  responsibility  of  the  author. 

An  opinion  tendered  is  that  some  of  the  basic  building  blocks  needed  to 
place  a  firm  foundation  under  detection  theory  appear  in  this  report.  The 
test  of  time  will  determine  any  credence  to  this  opinion. 

Though  the  reported  work  sets  forth  a  useful  mathematical  statement  of  the 
sufficiency  conditions  for  a  detection  system,  much  labor  must  still  be 
expended  to  further  explore  this  work  and  reduce  it  to  practice.  Only 
after  this  expenditure  of  effort  and  after  critical  scrutiny  by  those  more 
knowledgeable  than  the  author  in  detection  theory  can  the  significance  of 
this  work  be  properly  assessed. 


J.  Steranka,  Jr. 
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1.0 


INTRODUCTION 


An  apparatus,  a  machine  or  a  system  if  you  wish,  performs  in  accordance 
with  its  basic  sensory  capabilities.  That  which  it  seeks  to  obtain  is  in¬ 
formation  requested  by  the  system  proper;  that  which  it  does  obtain  varies 
with  information  climate  as  well  as  with  its  own  detective  capabilities. 

The  better  its  information-collecting  properties,  the  better  is  the  system 
total.  Detection  is  indeed  a  major  dictate  on  ultimate  system  performance. 

If  the  energy-sensing  medium  is  under  control,  as  in  active  sonar  or 
radar,  a  unique  signature  attached  thereto  eases  the  selection  of  relevant 
information  from  the  total  environment.  In  passive  detection  where  the 
source  under  observation  has  characteristics  peculiar  to  those  of  its  en¬ 
vironment,  a  detector  "tuned"  to  such  characteristics  often  does  well  in 

* 

following  source  behavior.  Under  these  circumstances,  there  is  very  often 
reasonable  certainty  the  received  quantity  is  relevant.  Passive  detection 
in  general,  however,  permits  only  some  weighted  measure  of  relevant  in¬ 
formation  in  the  received  signal-noise  quantity. 

In  the  limit,  moreover,  detection  systems  generally  involve  a  weighted 
measure  of  the  relevant  information  content  because  of  imperfections 
therein.  Here,  most  importantly,  the  internal  noise  associated  with  a 
detection  system  can  govern  uncertainty  in  signal  detection.  In  infrared 
and  stellar  detection  systems,  the  internal  noise  environment  is  so  ad¬ 
verse  at  times  to  rule  out  these  devices  for  given  applications.  Sometimes  • 

the  problem  is  so  severe  as  to  delay  system  development  until  detector 
state-of-the-art  improves. 

Confronted  with  detection  in  an  adverse  noise  environment,  any  one  sam¬ 
ple  of  the  signal-noise  quantity  then  has  little  meaning;  that  is,  any  one 
sample  tells  us  little  about  the  relative  contributions  of  signal  and  noise 
to  this  sample.  Invariably,  therefore,  a  large  population  of  signal-noise 

i 

samples  are  assembled,  and  inferred  therefrom  is  an  average  estimate  of 
"signal"  present.  In  the  limit,  this  becomes  the  integral  of  the  signal- 
noise  quantity.  "Signal"  detection  in  the  presence  of  noise,  therefore, 
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reduces  to  the  formation  of  some  statistical  estimate  of  signal  present  by 
observing  the  signal-noise  quantity  for  some  predetermined  time  interval. 

A  brief  scan  of  the  literature  reveals  quickly  an  abundance  of  attention  de¬ 
voted  to  the  detection  of  time-variant  signals  in  the  presence  of  noise. 
Analyses  have  proceeded  in  numerous  ways  through  the  use  of  various 
statistical  and  network  theory  tools,  with  varied  degrees  of  success. 

These  tools  we  might  say  comprise  the  area  of  "Statistical  Theory  of  Sig¬ 
nal  Detection.  " 

One  approach  which  appears  most  attractive,  as  it  clearly  commences 
with  a  statement  embodying  all  relevant  information  available  at  the  out¬ 
put  of  a  detector,  involves  the  formation  of  the  a  posteriori  probability  of 
a  signal  event  given  the  occurrence  of  a  signal-noise  event.  This  approach 
was  applied  successfully  to  the  radar  return  problem  some  ten  years  back 
by  Woodward.  *  Since,  additional  consequential  work  haB  been  done  there¬ 
on.  ** 


In  problems  as  radar  return,  the  time  shape  of  signal  sought  is  presumed 
known,  allowing  the  development  of  rather  definitive  statements  regarding 
the  sufficient  conditions  confronting  any  contemplated  detection  scheme. 
For  the  general  stochastic  signal  case,  however,  such  unavailable  a  priori 
knowledge  of  signal  time  shape  can  obstruct  final  definition. 

Yet,  the  clarity  of  the  a  posteriori  approach  as  developed  by  Woodward 
beckons  us  to  explore  this  method  for  the  general  stochastic  case. 

Such  exploration,  in  fact,  with  the  aid  of  series  representation  of 
stochastic  signals,  leads  to  a  basic  revelation  not  heretofore  treated  in 
detection  theory.  A  restatement  of  the  a  posteriori  probability  through 
series  signal  representation  clearly  shows  that  a  detection  system  is 

*  Reference  1  (Woodward). 

**  Reference  2  (Helstrom). 
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fundamentally  an  information  detector,  and  not  a  detector  of  signals.  In 
brief,  the  results  state  that  the  detection  system  can  only  provide  esti¬ 
mates  of  relevant  information  appearing  in  the  signal-noise  quantity;  in¬ 
ferences  drawn  from  these  estimates  in  a  subsequent  signal  processor 
will  say  something  about  signal  present.  This  latter  operation,  involving 
"guesswork"  on  the  information  measures,  has  nothing  to  do  with  signal 
detection,  however. 

Use  of  time  power  series  is  especially  employed  in  this  report  to  mathe¬ 
matically  demonstrate  the  aforesaid  statements.  Of  interest  is  the  fact 
that  the  results  so  obtained  clearly  express  the  sufficiency  conditions  for 
an  ideal  detector.  They  fall  out  in  a  form,  moreover,  which  are  suitable 
for  implementation.  The  results  agree  with  current  practices,  and  show 
them  to  be  approximations  to  the  ideal  requirements.  They  also  appear  in 
a  form  suitable  to  possible  systematic  improvements  to  present  practices. 

The  basic  information  detection  requirements  are  further  explored  through 
Fourier  series  expansion,  with  the  same  conclusive  success  as  with  power 
series.  These  results  alone,  together  with  their  product  representation, 
permit  treatment  of  a  broad  spectrum  of  problems. 

As  an  aside,  it  is  of  interest  to  point  out  that  the  posterior  statement  given 
through  Fourier-power  series  expansion  embodies  Woodward's  posterior 
statement  for  radar  return  as  a  special  case.  Amplitude -modulated  radio 
reception  is  satisfied  as  well.  Frequency  modulation  can  also  be  accom- 
tnodated,  though  a  form  of  Bessel  expansion  should  provide  a  more  useful 
Set  of  requirements.  This  latter  statement  makes  reference  to  the  fact 
that  the  work  in  this  report  shows  that  expansions  other  than  the  Fourier 
and  power  series  can  be  used  in  problem  solution.  The  physical  realiza¬ 
tion  of  the  ideal  requirements  ultimately  dictates  the  signal  expansion  which 
should  be  used. 

The  reported  work  centers  about  a  Gaussian  noise  environment,  and  as 
such  should  find  broad  utility.  Although  primary  attention  is  on  the 
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sufficiency  conditions  for  a  detection  system,  the  results  are  expressed 
in  a  form  which  explicitly  reflect  what  are  the  detection  system  require¬ 
ments  and  what  are  the  requirements  for  some  subsequent  signal  pro¬ 
cessing  system. 

As  an  illustrative  example,  a  signal  processing  system  for  the  realization 
of  maximum  likelihood  receives  attention.  The  posterior  statement  gen¬ 
erated  through  series  signal  representation  is  shown  to  lead  to  its  ready 
implementation. 

Added  attention  is  also  given  to  a  signal  processor  where  the  signal  sought 
is  approximated  by  a  step-like  function.  A  particular  interpretation  of 
the  constant -value  steps  in  the  signal  processor  provides  maximum  use  of 
detected  quantity. 

Though  the  work  lends  itself  to  direct  application,  numerical  examples 
are  not  treated  herein.  The  primary  viewpoint  taken  relates  to  passive 
detection;  the  results  equally  accommodate  signal  detection  problems  in 
active  systems,  however. 

In  summary,  the  reported  work  complements  present  detection  theory  by 
providing  a  useful  mathematical  statement  of  the  sufficiency  conditions 
for  systems  endeavoring  to  detect  time-variant  stochastic  signals  in  the 
presence  of  Gaussian  noise. 
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THE  A  POSTERIORI  PROBABILITY 


2.  0 


The  basic  approach  employing  the  a  posteriori  probability  as  set  down  by 
Woodward  for  time-variant  signals  is  first  presented.  A  discussion  re¬ 
garding  the  difficulties,  and  resolution  thereof,  in  this  approach  for 
stochastic  signals  then  follows.  With  the  basic  passive  signal  detection 
system  outlined  next,  the  general  solution  for  a  white  Gaussian  noise 
model  is  subsequently  presented.  Discussion  on  the  broad  import  of  this 
solution  and  its  extension  then  culminates  this  part  of  the  work. 


2.  1  BASIC  APPROACH* 

Fundamentally,  the  problem  of  signal  detection  can  be  expressed  as  the 
extraction  of  a  signal  quantity  from  a  signal-noise  quantity.  This  prob¬ 
lem  can  be  explicitly  stated  in  mathematical  form  through  the  a  posteriori 
probability  for  signal  detection. 

Development  of  the  general  a  posteriori  statement  for  such  a  two-variable 
case  is  derivable  directly  from  the  multiplication  law  of  probability,  by 
which  it  may  be  written 

P(X,  Y)  =  P(X)P(Y/X)=  P(Y)P(X / Y)  (2-1) 

which  reads  as  the  probability  of  the  occurrence  of  both  X  and  Y  is  equal 
to  both  the  probability  of  X  times  the  conditional  probability  of  Y  given  X, 
and  the  probability  of  Y  times  the  conditional  probability  of  X  given  Y. 

If  X  is  defined  as  the  signal  event  and  Y  as  the  signal-noise  event,  the  a 
posteriori  probability  for  signal  detection  then  is 

p,X/Y>  =  ,2-J) 


♦  Reference  1  (Woodward),  Chapter  4:  Statement  of  the  basic  approach  is 
borrowed  essentially  directly  from  this  text;  its  extension  thereafter, 
however,  is  a  sole  claim  of  the  author. 
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Since  in  the  detection  problem  Y  is  the  given  quantity,  equation  (2-2)  may 
be  written  in  reduced  form  as 

P(X/Y)  =  K  P(X)  P(Y/X)  =  K  P(X)  L(X)  (2-3)* 

where  P(Y)  is  absorbed  into  the  constant  K  which  itself  is  so  chosen  that 
the  conditional  posterior  probability  totals  to  unity  for  all  values  of  X. 


The  a  posteriori  probability  of  equation  (2-3)  is,  therefore,  a  function  of 
both  an  a  priori  statement  on  the  signal  event  X  and  the  conditional  state¬ 
ment  P(Y/X).  The  latter  conditional  statement  is  known  as  the  likelihood 
function,  so  designated  in  equation  (2-3)  by  replacing  P(Y /X)  with  L(X). 
This  is  not  a  probability  statement,  however,  since  it  is  treated  as  a 
function  in  X  for  a  given  Y. 


Assuming  noise  contribution  to  an  additive  signal-noise  quantity  Y  to  be 
Gaussian  with  zero  mean,  the  likelihood  function  for  an  arbitrary  sample 
of  Y  then  is 


L(X  )  =  K  exp 


(Y -X)‘ 


2cr 


N 


where  cr^  is  the  mean  squared  value  of  noise. 


(2-4) 


With  this  result  inserted  in  equation  (2-3),  we  can  deduce  therefrom  that 
one  such  sample  of  Y  tells  little  about  X  if  noise  is  severe.  Rather  than 
one  such  spot  sample,  we  might  then  ask  of  the  requirements  for  observing 
the  signal-noise  quantity  for  some  finite  time,  time  interval  (O,  T)  let  us 
say,  thereby  endeavoring  to  get  a  better  estimate  of  signal  present.  To 
answer  this  then  requires  formation  of  the  likelihood  function  for  this 
time  interval. 


To  develop  this  likelihood  function,  it  is  convenient  to  consider  band- 
limited,  white  Gaussian  noise,  that  is,  a  white  noise  frequency  structure 

*  The  constant  K  will  appear  throughout  the  work  having  different  values 
at  different  times,  but  always  chosen  such  that  the  probability  statement 
totals  to  unity  for  all  possible  events. 
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constrained  within  some  arbitrary  bandwidth  W.  Doing  this  permits  noise 
representation  over  time  interval  {O,  T)  by  a  finite  set  of  noise  samples, 
2WT  such  samples,  in  fact,  taken  every  1/2W  seconds,  as  permitted  by 
the  basic  sampling  theorem..*  Assuming  statistically  independent  samples 
permits  the  likelihood  function  for  interval  (O,  T)  to  be  written  as 


2WT 

L<x>  ■  L1 


=  K  exp 


n' 


2WT 

E 

n=l 


t*  A 

JT  e 

(Y  -X  )2 
n  n 

Cn=l  CXP 

2"n 

(Y  -X  )2  , 
N  n  n' 

2"n 

(2-5) 


Since  the  noise  samples  are  an  orthogonal  set,  as  per  the  sampling  theorem, 
the  sum  appearing  in  equation  (2-5)  can  be  written  as 


2WT  ,v  Y  .2  T 

•  2  — ” . y  "  =i?  f  (y-x)2dt 

n=l  2a^  o*t> 


(2-6)#* 


where  Nq  is  equal  to  CT^  /W,  or  the  mean  noise  power  per  unit  bandwidth. 
Thus,  the  likelihood  function  of  equation  ( Z  —  5i)  can  be  expressed  as 

rT 


Li(X)  =  K  exp 


R  (  (Y-X)2dt 
o  Jo 


(2-7) 


and  the  a  posteriori  probability  for  signal  event  X  during  time  interval 
(O,  T)  of  equation  (2-3)  may  be  written  as  • 


P(X/Y)  =  K  P(X)  exp 


i  |  (Y-X)2dt] 
O  Jo 


(2-8) 


Expansion  of  the  exponent  term  in  this  equation  permits  the  reduced  form 


T  T 

P(X/Y)  =  K  P(X)  exp  -^j  f  X2dt  exp  |  fxYdt 

O  Jo  O  Jo 


(2-9) 


*  Reference  3  (Shannon)  and  Reference  4  (Shannon  and  Weaver)  are 
recommended. 

**  Reference  1  (Woodward),  Chapter  2. 
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2 

where  the  Y  term  of  the  integrand,  with  Y  being  given,  is  absorbed  into 
the  constant  K. 

Thus,  from  the  final  posterior  statement  in  equation  (2-9),  the  only  oper¬ 
ation  which  need  be  performed  on  the  signal-noise  quantity  Y  involves  the 
integral  of  the  product  XY  over  interval  (O,  T).  This  is  indeed  an  irre¬ 
versible,  noise -stripping  operation  on  Y,  in  essence  being  the  correlation 
of  Y  with  all  X  over  interval  (O,  T).  Realization  of  this  integral  then  re¬ 
presents  a  just  sufficient  solution  to  form  the  a  posteriori  probability  of  X 
for  any  a  priori  P(X)  chosen  for  X. 

Formation  of  the  XY  integral,  moreover,  constitutes  the  sufficiency  con¬ 
ditions  for  a  detection  system.  Selection  of  P(X)  and  determination  of  the 
2 

X  integral  in  equation  (2-9)  are  not  requirements  of  the  detection  system 
itself,  but  requirements  of  a  subsequent  processing  system,  wherein  de¬ 
tection  system  output  may  be  interpreted  as  one  pleases.  If  P(X)  is  known, 
then  the  posterior  statement  could  be  formed  exactly  (if  given  integral  XY, 
that  is);  if  some  arbitrary  form  of  P(X)  is  presumed,  then  the  resulting 
posterior  statement  is  only  as  good  as  the  closeness  between  the  presumed 
P(X)  and  the  actual  case.  In  neither  event,  however,  does  such  data  in¬ 
terpretation  impose  additional  demands  on  the  detection  system  itself;  it 
still  need  only  form  the  integral  of  XY  over  interval  (O,  T). 

In  the  general  passive  detection  case,  time  shape  of  X  is  not  known;  hence, 
,the  sufficiency  conditions  expressed  through  equation  (2-9)  are  not  directly 
realizable,  that  is,  the  integral  of  XY  is  not  directly  formable.  Resolution 
of  this  point,  in  fact,  is  the  crux  of  the  reported  work. 

As  an  introductory  exercise  to  subsequent  work,  let  us  at  this  time  con¬ 
sider  a  known  time  shape  for  X;  let  this  known  shape  be  some  constant 

value  XQ  over  interval  (O,  T).  For  this  simple  case,  equation  (2-9)  re¬ 

duces  to 

X  *T )  (  2X  rT 

P(X/Y)  =  K  P(X)  exp  -  exp  -jA  Ydt 

o  I  o  •'o 


(2-10) 
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The  sufficiency  conditions  for  a  detection  system  then  reduce  to  a  simple 
integral  of  the  signal-noise  quantity  Y. 

If  P(X)  is  assumed  to  be  a  uniform  distribution,  examination  of  equation 
(2-10)  shows  the  most  probable  value  of  X  equal  to  the  average  value  of  Y 
over  interval  (O,  T).  Such  an  assumed  a  priori  distribution  is  equivalent 
to  the  application  of  maximum  likelihood,  of  course,  which  then  gives  the 
same  solution,  a  solution  also  long  since  intuitively  derived  by  the  elec¬ 
tronic  system  designer. 

In  passing,  we  should  make  an  observationregardingthe  value  of  observing 

the  signal-noise  quantity  for  interval  (O,  T)  as  opposed  to  a  spot  sample. 

This  can  be  done  by  comparing  the  variances  of  their  likelihood  functions 

for  the  constant  signal  case.  For  a  spot  sample,  equation  (2-7)  gives  a 
2 

variance  of  oM ;  for  observation  over  interval  (O,  T),  equation  (2-10)  gives 

N  2  - 

a  variance  of  N  / 2T,  or  a.,/2WT.  Thus,  by  this  measure,  we  see  an  im- 
O  IN 

provement  in  signal  estimation  of  2WT. 


2.  2  DISCUSSION  OF  SUFFICIENCY  CONDITION  REALIZABILITY 

As  implied  heretofore,  formation  of  the  XY  integral  over  a  chosen  inter¬ 
val  (O,  T)  represents  the  sufficiency  conditions  for  a  detection  system. 
Formation  of  this  integral  permits  subsequent  development  of  the  poste¬ 
rior  statement  regarding  ^  ignal  present  in  this  interval.  To  realize  this 
integral  in  passive  detection,  however,  poses  some  difficulty,  as  a  priori 
knowledge  of  signal  time  shape  is  not  generally  available.  Direct  im¬ 
plementation  appears  out  of  the  question,  since,  for  a  time -variant 
stochastic  signal,  the  integral  need  be  formed  for  an  infinity  of  X  signal 
time  shapes. 

Yet,  passive  detection  systems  are  in  existence  today,  some  of  which  do 
an  admirable  job  in  picking  a  signal  from  a  morass  of  noise.  We  might 
then  conclude  from  such  evidence  that  these  systems  must  in  some  way  ap¬ 
proximate  the  sufficiency  conditions  set  forth.  If  this  is  indeed  true,  it 


2-5 


should  be  possible  to  relate  present  successful  practices  to  the  ideal  set 
of  requirements.  Moreover,  should  such  a  relationship  be  realized,  we 
might  expect  therefrom  the  direction  to  proceed  to  obtain  even  better  ap¬ 
proximations  to  the  ideal  set.* 

To  show  the  foregoing  desired  objectives  to  be  realizable,  we  must  delve: 
in  the  subtle  facets  of  troublesome  integral  XY.  More  specifically,  we 
must  examine  ways  of  realizing  this  integral  for  X  unknown:  a  noble  task 
indeed. 

It  will  be  shortly  shown  that  prior  knowledge  of  X  need  not  be  available  to 
form  this  integral,  at  least  in  principle,  if  we  are  judicious  in  the  way  an 
arbitrary  X  is  represented.  The  basic  argument  underlying  resolution  of 
this  integral  appears  in  Appendix  C  of  this  report. 

Regarding  the  unknown  X  itself,  we  seek  a  more  useful  way  of  representing 
this  quantity,  a  representation  which  would  ultimately  permit  solution  of 
the  XY  integral.  We  see  difficulty  in  forming  this  integral  directly  only 
because  X  is  envisioned  as  an  infinity  of  unknown  time  samples.  In  a 
way,  the  task  would  seem  less  difficult  if  an  arbitrary  X  were  represented 
by  a  finite  set  of  samples  through  use  of  the  sampling  theorem;  formation 
of  the  required  integrals  involve  sin  wt/t  type  functions,  however,  over 
time  minus  infinity  to  plus  infinity,  causing  us  to  lose  enthusiasm  in  this 
representation. 

In  this  search  for  a  useful  form  of  X,  we  must  always  keep  in  mind  the 
practicality  of  implementing  the  required  integration  resulting  from  any 
new  representation  of  X.  Realizing  this,  the  author  cannot  help  but  reflect 


*  This  is  not  idle  talk,  for  the  situation  here  is  analogous  to  the  problem 
of  realizing  better  performance  with  a  serial,  general-purpose  digital 
computer  through  better  signal  representation.  This  computer  problem 
was  resolved  through  reasoning  similar  to  the  above.  We  refer  to 
Reference  5  and  6,  even  though  they  are  only  available  under  controlled 
distribution.  The  basic  philosophy  expounded  therein,  however,  is 
lightly  treated  in  Appendix  A  of  this  report. 
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back  to  the  signal  representation  used  in  improving  serial,  general-pur¬ 
pose  computer  operation,*  Therein,  a  power  series  in  time  was  employ¬ 
ed  as  the  general  form  to  represent  a  signal,  because  of  its  ease  in  im¬ 
plementation.  Why  not  then  assume  the  signal  sought  in  the  detection 
problem  to  be  also  representable  by  this  form?  This  form  can  be  put  to 
good  use  in  most  cases.  Its  value  lies  in  the  way  its  coefficients  are 
chosen,  and  some  extremely  powerful  selections  can  be  made  there¬ 
with.  ## 


If  X  is  represented  by  a  power  series  in  time  with  arbitrary  coefficients, 
the  solution  of  the  general  XY  integral  is  direct  —  embarassingly  di¬ 
rect  to  say  the  least.  Its  implementation,  moreover,  is  straightforward, 
involving  only  successive  integrals  of  Y.  The  arbitrary  coefficients  need 
not  be  prestated.  The  successive  integrals  of  Y  measure  as  best  possible 
the  relevant  information  content;  these  coefficients  then  can  be  determined 
from  these  measures  at  a  later  date. 

We  shall  shortly  show  formally  the  simplicity  to  this  general  solution. 
Presently,  we  first  choose  to  set  forth  a  model  for  the  general  passive  de¬ 
tection  system  for  use  in  subsequent  discussion. 


2.  3  PASSIVE  DETECTION  SYSTEM 

A  model  for  a  general  passive  detection  system  is  depicted  in  figure  2-1. 
Therein,  the  environment  to  which  a  detector  is  exposed  is  called  the  inr 
coming  energy  environment.  The  signal  energy  represents  that  relevant 
information  requested  by  the  system  functions;  the  remainder,  which  might 
pass  through  the  detector,  is  termed  background  energy  (or  signal  noise 
environment). 

A  filter  system  is  inserted  between  the  incoming  energy  environment  and 

the  detector  to  accommodate  those  systems  employing  some  sort  of  "space 

*  Appendix  A  of  this  report. 

**  Appendix  B  of  this  report. 
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Figure  2-  1. 


Passive  Detection  System. 
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filtering"  prior  to  detection.  This  might  range  anywhere  from  simple 
amplitude  modulation  to  some  form  of  random  filtering.  The  filter  char¬ 
acteristic  R(t)  is  defined  such  that  its  output  is  equal  to  the  incoming  en¬ 
ergy  time  function  multiplied  by  R(t).  In  subsequent  work,  R(t)  is  always 
assumed  to  be  known  explicitly. 

The  detection  system  is  shown  as  being  comprised  of  both  a  detector  and 
preprocessor.  The  detection  system  is  assumed  to  have  consequential 
internal  noise.  A  preprocessor  is  included  as  an  integral  part  of  the  de¬ 
tection  system  to  emphasize  that  provision  of  some  unstated  measure  of 
incoming  energy  behavior  by  itself  constitutes  an  incomplete  detection 
system;  it  must  also  state  as  best  possible  that  portion  of  relevant  infor¬ 
mation  contained  in  its  output  structure.  This  latter  requirement  falls 
upon  the  preprocessor. 

•In  a  strict  sense,  preprocessor  is  only  required  to  provide  a  measure  re¬ 
flecting  that  portion  of  incoming  energy  appearing  in  total  detector  output 
quantity.  That  is,  all  incoming  energy  is  treated  as  signal  energy  by  the 
preprocessor,  and  noise  is  constrained  to  mean  detector  internal  noise. 
When  system  functions  request  a  signal  with  specific  information  struc¬ 
ture,  the  preprocessor  can  be  burdened  with  added  special  requirements. 
Here,  background  energy  and  detector  internal  noise  combine  to  represent 
total  noise  environment. 

A  signal  processor  is  shown  distinct  to  the  preprocessor  of  the  detection 
system.  This  is  done  since  the  signal  processing  operation  can  be  chosen 
independent  of  the  detection  system.  The  signal  processor  task  is  to  take 
preprocessor  measures  of  relevant  information  content  in  detector  output 
and  construct  therefrom,  or  infer  therefrom,  the  nature  of  the  signal 
present.  The  signal  processor  function  might  be  in  the  form  of  a  prob¬ 
ability  statement  with,  or  without  some  decision-making  operation,  or  in 
the  form  of  an  inferred  signal  as  a  function  of  time.  System  functions  will 
generally  dictate  the  nature  of  signal  processor  operation. 
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GENERAL  SOLUTION  USING  POWER  SERIES  (BAND-LIMITED 
WHITE  GAUSSIAN  NOISE) 


To  derive  the  general  solution  for  a  time-variant  stochastic  signal  using 
power  series  expansion  we  remain  with  the  noise  environment  which  is 
band-limited,  white  Gaussian  with  zero  mean.  In  terms  of  the  passive  de¬ 
tection  system  given  in  figure  2-1,  all  incoming  energy  can  be  assumed 
to  constitute  signal,  and  noise  to  be  internal  detector  noise;  a  filter  system 
is  not  employed,  or,  as  per  figure  2-1,  filter  transfer  characteristic  is 
unity.  The  object  is  to  determine  requirements  for  an  ideal  preprocessor. 


Deeming  spot  samples  of  detector  output  to  be  inadequate,  we  then  choose 
observation  of  detector  output  quantity  for  some  time  interval  (O,  T)  as 
the  basic  detection  mode  of  operation.  For  the  noise  environment  chosen, 
previous  equation  (2-9)  constitutes  the  posterior  statement  regarding  sig¬ 
nal  detection.  To  aid  the  reader,  this  equation  is  repeated  below: 


P(X/Y)  =  K  P(X)  exp 


-B  fx2* 

exp 

2  fT 

fi  XYdt 

O 

O 

(2-11) 


We  shall  now  proceed  to  restate  this  expression  in  more  meaningful  form 
by  representing  the  signal  sought  through  power  series  expansion. 


First,  X  is  written  as  a  power  series  in  time  over  interval  (O,  T): 

CD 

X(0,  T)  =  Ao  +  Ajt  +  A2t2  +  •  *  •  +  Aktk  +  •  •  •  =2  Antn  (2-12) 

n=o 

where  the  A's  are  arbitrary  constants  left  to  our  choosing.  *  Using  this 
form  of  X  in  equation  (2-11)  results  in 


*  Definition  of  the  A's  could  be  made  a  priori  as  per  Appendix  B,  or  made 
a  posteriori  on  the  basis  of  maximum  likelihood  (Section  3.0),  most 
probable  value  through  use  of  an  a  priori  distribution,  or  some  other 
method  suitable  to  particular  system  application.  They  are,  however, 
constants  which  can  be  treated  as  an  arbitrary  set  insofar  as  the  de¬ 
tection  system  is  concerned. 
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-m+n+1 


P(X/Y)  =  K  P(X)  exp  .-f,  Y,  £ 

v  o  n=o  m=o 

exp{?j  /YS  vndt} 


(2-13) 


o  o  n=o 


Since  P(X)  and  the  A's  are  to  our  choosing,  only  the  integral  term  needs 
resolution.  This  is  the  same  integral  we  were  confronted  with  in  Appen¬ 
dix  B  of  this  report,  wherein  it  was  directly  solved  through  integration 
by  parts.  Proceeding  as  per  Appendix  B,  we  may  write 


-o[jrTH*A>[T( 


[ i Ydt] + A*  [T  ( Ydt  ■  [  iYitZ ] 

[ T2  /Yd.-ZT  /Y«2+2  jy/vatO 
Lo  oo  ooo  ^ 

.  ,  A  f  /  »rn+l  kl  ^(k+l-m) 

Mfel'  (k+l -m)!  T 


rm[Y]] 


(2-14) 


whereSTm[Y]  represents  the  mth  integral  of  Y  over  interval  ('O',  T).  This 
solution  can  be  written  in  more  compact  form  as 


/  YE  V*  -2>nE  l-»m+1  CTT  t'»-»+1's-"'[y] 
o  n=o  n=o  m  =  l  '  '  (2-15) 

Now,  P(X)  can  be  expressed  in  general  form  in  terms  of  the  A’s,  that  is, 


P(X)  =  P(Aq,  Ar  A2,  •  •  •  ,  Ak,  •  •  • )  =  P(|A|) 
Utilizing  equations  (2-15)  and  (2-16)  in  equation  (2-13)  gives 


(2-16) 


2-11 


P(X/Y)  =  P({A}  /Y) 


\  ,  “  Tm+n+l 

KP({A|)  .*p  4  ES  AmA„  fcnT  ! 


o  n-o  m=o 


“  n+1 

-*{»  2XL<->mtl  rar  T 

f  o  n=o  m=l  ' 


(n-m+1) 


S~m  [yJ  |  (2-17) 

This  equation  represents  a  useful  statement  of  the  a  posteriori  probability 
of  a  signal  X  with  unknown  time  shape  in  the  presence  of  band-limited, 
white  Gaussian  noise  with  zero  mean.  For  any  chosen  P(|a|)  and  |a|, 
only  formation  of  the  successive  integrals  of  the  signal-noise  quantity  Y 
over  interval  (O,  T)  are  needed  to  realize  the  full  posterior  statement  for 
signal  detection.  The  requirements  for  an  ideal  preprocessor  then  become 
the  formation  of  these  integrals,  that  is,  formation  of 


*  T  T  t  ,  T  t  t 

E  f  m[Y]  — /Ydt  ;  /  /  Ydt2  ;  //./ 
m=l  o  o  o  o  o  o 


Ydt  ;  Etc. 


(2-18) 


The  immediate  realizability  of  the  aforesaid  integrals  should  be  apparent, 
though  we  would  hardly  choose  to  realize  an  infinite  set. 


Note  specifically  what  power  series  representation  for  time-variant 
stochastic  signals  has  done  to  the  detection  system  requirements.  It  has 
removed  determination  of  the  power  series  arbitrary  coefficients  as  a  re¬ 
quirement  thereof,  leaving  this  determination  to  some  subsequent  signal 
processing  system.  This  is  indeed  valid  since  the  equation  (2-18)  inte¬ 
grals  measure  as  best  possible  the  relevant  information  in  detector  out¬ 
put  quantity.  The  arbitrary  coefficients  can  be  determined  from  these 
measures  by  any  means  we  choose.  Whatever  method  chosen,  it  will  not 
be  impaired  by  a  shorted  supply  of  information,  for  the  infinite  set  of 
integrals  reflect  all  that  can  be  obtained. 

Request  for  direct  measures  of  a  priori  stated  coefficients  by  a  detection 
system  would  represent  more  than  sufficient  conditions.  This  would  not 
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only  be  a  request  for  the  best  measure  of  relevant  information  appearing 
in  detector  output  quantity,  but  also  a  request  for  some  inferences  to  be 
drawn  therefrom.  The  latter  is  not  a  task  of  basic  information  detection 
which  is  the  only  responsibility  of  a  detection  system,  but  a  task  involving 
"guesswork"  on  the  information  measures  obtained  which  is  the  respon¬ 
sibility  of  some  subsequent  signal  processing  system. 

We  might  surmise,  therefore,  that  lack  of  appreciation  of  this  basic  point 
has  obstructed  treatment  of  time -variant  stochastic  signals  by  the  theo¬ 
rists  in  the  field  of  signal  detection. 


2.  5  APPLICATION  OF  GENERAL  SOLUTION  OBTAINED  THOUGH 
POWER  SERIES 

Using  power  series  representation  for  unknown  signal  X,  the  require¬ 
ments  for  an  ideal  preprocessor  become  the  formation  of  an  infinite  set  of 
successive  integrals  of  detector  output  over  observation  interval  (O,  T), 
as  per  equation  (2-18).  The  integrals  themselves  are  directly  realizable. 
In  practice,  moreover,  signal  behavior  is  often  such  that  formation  of 
the  first  integral  alone  is  sufficient  to  satisfactorily  reflect  its  detection 
in  presence  of  noise.  Here,  length  of  observation  interval  T  is  the 
dominant  factor,  and  its  value  is  so  chosen  that,  for  all  practical  pur¬ 
poses,  the  signal  is  constant  over  the  observation  interval.  Sometimes 
other  system  requirements  dictate  a  value  of  T  such  that  signal  must  be 
considered  time-variant  over  the  interval,  requiring  formation  of  the  first 
two  or  first  three  terms  of  the  integral  set.  It  can  be  generally  stated, 
however,  that  requirements  for  an  ideal  preprocessor,  as  given  in  equa¬ 
tion  (2-18),  can  usually  be  adequately  approximated  by  but  a  few  of  the 
integral  measures  at  most  (that  is,  if  the  power  series  form  is  an  efficient 
r  epr  e  sentatiOn) , 

Interesting  enough,  realization  of  the  required  integrals  by  a  preprocessor 
in  no  way  affects  subsequent  signal  processing.  The  arbitrary  constants 
{A}  of  the  time  power  series  representing  signal  X  remain  to  be  chosen 
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therein  as  one  pleases.  The  same  is  also  true  with  regard  to  the  a 
priori  distribution  for  X. 

In  passing,  it  is  of  importance  to  relate  the  significance  of  these  state¬ 
ments  with  regard  to  applications  where  signal  detected  is  recorded  (or 
transmitted  outside  the  system)  for  use  elsewhere.  Its  further  use  might 
take  many  varied,  and  sometimes  yet  to  be  specified  forms.  Retention  of 
relevant  information  in  this  basic  integral-measure  form  permits  these 
subsequent  operations.  They  in  no  way  impair  freedom  in  operation 
thereafter. 

Added  reference  is  also  made  to  space  systems,  wherein  signal  detected 
is  ultimately  transmitted  back  to  the  earth  in  digital  form.  Here,  con¬ 
straints  on  such  data  transmittal  are  usually  so  severe  that  maximization 
of  data  compression  is  of  primary  concern.  Straightforward  sampling  of 
signal  quantity  simply  involves  much  too  high  bit  rates.  It  then  becomes  a 
problem  of  finding  a  signal  format  which  lendB  itself  to  the  most  efficient 
coding  of  signal  data. 

A  little  thought  on  the  integral  measures  of  equation  (2-18)  suggests  these 
measures  as  a  satisfactory  signal  format.  Rather  than  a  set  of  sampled 
amplitudes,  we  might  use  these  measures  to  represent  a  signal  as  a  power 
series  for  some  time  interval.  In  fact,  related  work  in  this  area  by  the 
author  indicates  this  to  be  highly  effective.  In  such  cases,  signal  is  repre¬ 
sented  by  an  integral  or  a  finite  integral  set,  as  per  equation  (2-18),  of  de¬ 
tector  output  quantity.  Only  these  integrals  are  transmitted  back  to  the 
earth.  Spectral  behavior  of  signal  and  other  system  requirements  deter¬ 
mine  length  of  observation  interval  T  and  number  of  integral  measures, 
with  these  so  chosen  that  signal  data  bits  per  unit  time  are  a  minimum  for 
transmittal.  Such  signal  representation  for  transmittal,  moreover,  can 
be  advantageous  whether  or  not  the  noise  environment  is  of  consequence. 
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GENERAL  SOLUTION  USING  FOURIER  SERIES  (BAND-LIMITED 
WHITE  GAUSSIAN  NOISE) 


To  show  that  signal  representation  through  power  series  is  not  the  only 
form  of  expansion  which  leads  to  a  general  solution,  we  now  show  the  re¬ 
quirements  for  an  ideal  preprocessor  if  the  Fourier  series  expansion  is 
employed  (though  the  Fourier  series  is  embodied  in  the  general  power 
series).  The  noise  model  remains  as  band-limited,  white  Gaussian  noise. 


Suppose  the  arbitrary  signal  X  is  represented  by  the  general  Fourier 
series  over  the  observation  interval  as 


X(Q,  T)  =  £  Fn  cos  (5S.  t  +  en) 


-  (" a  sin  t  +  b  cos  1 1 

HLn  T  n  T  J 


(2-19) 


where  the  F's  (or  the  a's  and  b's)  are  arbitrary  constants,  whose  defini¬ 
tion  can  be  stated  at  a  later  time  (the  argument  used  on  power  series 
applies  directly).  If  this  representation  of  X  is  inserted  into  the  general1 
posterior  statement  for  band-limited,  white  Gaussian  noise,  as  given  in 
equation  (2-9)  and  (2-11),  we  see  the  XY  integral  to  decompose  such  that 
the  requirements  for  an  ideal  preprocessor  become 


(2-20) 


(2-21) 
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Thus,  with  the  signal  expressed  as  a  Fourier  series,  the  ideal  preproces¬ 
sor  must  provide  an  infinite  set  of  filters  matched  to  the  sinusoids  of  the 
harmonics  representing  the  signal.  (Note  that  need  for  synchronous  de¬ 
tection  is  embodied  in  these  ideal  requirements. ) 

Since  the  Fourier  series  representation  can  often  be  better  used  to  ap¬ 
proximate  a  signal  where  similar  approximation  through  power  series  is 
inefficient,  and  vice  versa,  the  two  sets  of  requirements  for  an  ideal 
preprocessor,  as  given  by  equations  (2-18)  and  (2-20)  or  (2-21),  present 
the  system  designer  with  formidable  guides  by  which  to  synthesize  his 
system. 

The  comments  regarding  application  of  the  power  series  solution  in  sec¬ 
tion  2.  5  apply  as  well  to  the  Fourier  series  solution  given  above. 


2.  7  SPECIAL  CASE  OF  FOURIER  SERIES  SOLUTION 


It  is  of  interest  to  examine  a  special  case  of  the  Fourier  series  solution. 
Reference  is  to  the  case  where  the  signal  sought  contains  only  one  fre¬ 
quency  and  is  of  constant  amplitude,  that  is,  the  signal  can  be  represented 
by 

X(0,  T)  =  F  cos  (ut  +  0)  (2-22) 

The  equation  (2-11)  posterior  statement  then  becomes 

(  -r.2  T 

P(X/Y)  =  K  P(X)  exp  j  -  /  cos4  (w  t  +  6)dt 

(  o  o 


exp 


|p  f  Ycos(w  t  +  0)dt 
o  o 


(2-23) 


We  see  equation  (2-23)  is  identical  with  Woodward's  posterior  statement 
for  the  radar  return  problem  for  known  signal  amplitude.  Of  course, 
equation  (2-23)  is  the  posterior  statement  for  exact  time  behavior  of  sig¬ 
nal  — —  phase  included.  Suppose  we  are  only  interested  in  whether  signal 
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is  present.  If  phase  is  unknown,  posterior  statement  for  presence  of  sig¬ 
nal  requires  integration  of  equation  (2-23)  over  all  possible  8.  Assuming 

a  uniform  distribution  for  8  then  leads  to  the  modified  Bessel  form  I  as 

o 

the  requirement  for  the  preprocessor. 

Incidentally,  if  the  envelope  of  equation  (2-22)  is  time -variant,  power 
series  representation  of  envelope  behavior  leads  to 

T  T  t  , 

Jy  cos  ( tit  t  +  8)  dt  ;  u  Y  cos(wt  +  8)dt  ; 

T  t  t  , 

(  f  C  Y  cos(<ot  +  8)dt  ;  Etc.  (2-24) 

o  o  o 

as  the  requirements  for  an  ideal  preprocessor.  Here  again,  the  posterior 
statement  is  for  both  amplitude  and  phase.  Integration  of  this  posterior 
statement  over  all  0  then  results  in  a  solution  suitable  to  Woodward's 
radar  return  work  on  signal  amplitudes  which  are  time-variant.  It  is,  in 
fact,  a  more  direct  solution.  This  same  solution  is  also  applicable  to 
amplitude -modulated  radio  reception  in  general. 


2.  8  PRE -DETECTION  FILTER  CONSIDERATIONS  (BAND-LIMITED 
WHITE  GAUSSIAN  NOISE) 

The  characteristics  R(t)  of  a  pre -detection  filter  were  defined  in  section 
2.  3,  as  per  figure  2-1,  such  that  its  output  was  given  by 

XR  =  X  R(t)  (2-25) 

If  X  and  R(t)  both  are  represented  by  a  power  series  in  time,  the  require¬ 
ments  for  an  ideal  preprocessor  operating  in  band-limited,  white  Gaussian 
noise  are  those  given  by  equation  (2-18):  an  infinite  set  of  successive  in¬ 
tegrals  of  detector  output  over  the  observation  interval. 

If  X  is  represented  by  a  power  series  and  R(t)  by  a  Fourier  series,  or  vice 
versa,  the  requirements  for  an  ideal  preprocessor  become 
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jh  / Ycos(^t+  en)dt  ;  jh  /  /  Ycoa(?yi 

n^o  o  TT^o  o  o 

oo  T  t  t  ^  ~ 

V  /  J  /  Ycosffi-  t  +  en)dt3;  Etc. 

y?*  a  y*  y\  r\ 


t  +  8n)dt2; 


(2-26) 


This  solution,  incidentally,  also  satisfies  the  case  where  a  signal  is  re¬ 
presented  by  the  product  of  a  Fourier  series  with  a  power  series. 


If  both  X  and  R(t)  are  represented  by  a  Fourier  series,  the  requirements 
for  an  ideal  preprocessor  are 

V  2  / Ycos(^t  +  0  )  cos(^2-t  +  0  )dt  (2-27) 

n=o  m=o  o 

The  preprocessor  requirements  given  through  equations  (2-18),  (2-26) 
and  (2-27)  then  represent  a  set  of  design  guides  which  should  cover  most 
cases  arising  in  practice. 


One  case  not  covered  by  the  above  set,  however,  is  when  the  pre-detection 
filter  frequency  modulates  the  signal.  Writing  the  output  of  the  pre-de¬ 
tection  filter  for  this  frequency  modulation  as 

XR  =  a(t)  cos  f(t)  (2-28) 

and  expressing  a(t)  as  a  power  series  leads  to  the  requirements  for  an 
ideal  preprocessor  concerned  with  the  general  case  of  frequency  modula¬ 
tion  as 


T  T  t  _  T  t  t, 

fYcosf(t)dt  ;  /  fYcosf(t)dt£  ;  J  j  J Ycosf(t)dt  ;  etc.  (2-29) 

*o  o  o  o  o  o 


In  each  of  the  above  cases,  explicit  knowledge  of  R(t)  is  presumed  known; 
the  ideal  requirements,  therefore,  have  embodied  therein  a  request  for 
synchronous  detection. 
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2.9 


GENERAL  GAUSSIAN  NOISE  CONSIDERATIONS 


Though  the  general  solutions  heretofore  obtained  were  for  Gaussian  noise 
involving  independent  samples,  it  can  be  readily  extended  to  include  cor¬ 
related  Gaussian  noise  with  some  additional  labor.  First,  however,  it  is 
necessary  to  perform  a  transformation  on  the  noise  function  in  order  to 
generate  a  set  of  independent  noise  samples.  It  can  then  be  shown*  that 
the  posterior  statement  for  signal  detection  may  be  written  asi 


P(X/Y)  =  K  P<X)  exp 


-  j  f  XQdt  J  exp  j  f  YQdt  | 


where  Q  i-s  defined  by  the  integral  equation 


(2-30) 


T 

XR(t)  =  |  %(t-s)  Q(s)ds  (2-31) 

with  0N  being  the  noise  autocovariance.  From  equation  (2-30),  we  see  the 
sufficiency  conditions  for  the  preprocessor  become  the  generation  of  in¬ 
tegral  YQ. 


Most  often  0j^,  or  its  integral,  is  well  behaved  permitting  its  representa¬ 
tion  in  Taylor's  form  of  the  time  power  series.  With  0^  so  represented, 
and  Q  assumed  in  the  form  of  representation  chosen  for  XR(t),  equation 
(2-31)  should  generally  lend  itself  to  ready  solution.  This  being  so,  the 
requirements  for  an  ideal  preprocessor  are  again  dependent  on  the  repre- 
sentation  chosen  for  XR(t),  being  either  those  in  equation  (2-18),  (2-26), 
or  (2-27)  for  power  and  Fourier  series  expansions  of  X  and  R(t). 


2.  10  A  FEW  COMMENTS  ON  SERIES  SOLUTIONS 

’A  close  examination  of  the  series  solutions  presented  shows  their  approx¬ 
imations  to  represent  methods  practiced  today.  We  leave  discussion  of 
these  approximations  to  the  reader,  who  is  probably  better  versed  in 

*  Based  on  Reference  2  (Helstrom),  Chapter  4. 
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current  practices.  A  clear  statement  of  these  approximations,  as  per 
our  solutions,  is  in  itself  a  successful  accomplishment.  Note  the  logical 
structure  of  these  series  solutions,  moreover;  if  the  presently  employed 
approximation  is  inadequate,  they  clearly  express  the  next  best  approxima¬ 
tion  which  can  be  made. 

It  should  be  emphasized  that  the  most  important  analytical  point  developed 
is  that  the  XY  integral,  which  defines  the  sufficiency  conditions  for  a 
preprocessor,  is  directly  solvable  if  the  signal  X  is  expressed  as  a  series 
expansion  in  time  with  arbitrary  coefficients.  Determination,  or  selec¬ 
tion  of  these  arbitrary  coefficients  is  a  task  for  the  signal  processing  sys¬ 
tem,  and  not  a  task  of  the  preprocessor. 

Though  only  Fourier  and  power  series  are  treated  herein,  one  is  not  re¬ 
stricted  to  such  signal  representation.  Many  attractive  expansions  are 
available,  especially  those  which  are  formed  from  an  orthogonal  set  of 
functions.  The  problem  application  will  ultimately  determine  the  best 
type  of  expansion  to  be  employed  for  solution.  Use  of  the  Bessel  form, 
for  example,  appears  to  offer  advantage  for  frequency  modulated  signals 
as  well  as  certain  pulse  type  signals. 

From  a  practical  point  of  view,  that  form  of  expansion  which  converges 
most  rapidly  for  the  type  of  signal  to  be  accommodated  and  whose  terms 
lend  themselves  to  ready  implementation  should  be  chosen.  Rapid  con¬ 
vergence  implies  only  a  few  terms  of  the  expansion  need  be  implemented 
to  realize  a  good  approximation  to  the  ideal  set  of  requirements. 

Use  of  the  a  posteriori  approach  not  only  justifies  itself,  but  also  strongly 
recommends  itself  as  the  approach  to  be  employed  on  signal  detection 
problems  in  general.  It  directly  identifies  the  Sufficiency  conditions  for 
detection,  which  can  tnen  be  readily  reduced  to  usable  form  through  series 
expansion  for  signal  representation. 
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3.0 


MAXIMUM  LIKELIHOOD  FOR  POWER  SERIES  REPRESENTATION 


Though  not  interested  in  signal  processing;,  per  se,  the  simplicity  of  real¬ 
izing  the  maximum  likelihood  method  through  series  representation  for 
time-variant  stochastic  signals  causes  us  to  devote  consequential  attention 
to  its  implementation.  It  also  serves  as  an  example  of  numerous  methods 
by  which  the  arbitrary  coefficients  of  the  series  expansions  employed 
could  be  obtained. 

Maximization  of  the  likelihood  function  appearing  in  the  posterior  state¬ 
ment  for  signal  detection  implies  that  all  X  signal  time  shapes  are  equally 
likely  to  occur  during  interval  {O,  T),  that  is,  any  set  of  arbitrary  con¬ 
stants  for  an  expansion  of  X  are  equi-probable.  The  a  priori  X  probability 
is  then  a  uniform  distribution  representable  by  some  arbitrary  constant. 
This  constant  itself  can  always  be  absorbed  by  the  all  encompassing  K 
which  accompanies  each  of  the  posterior  statements. 

Thus,  assuming  equi-probable  a  priori  probability  permits  a  reduced 
form  of  the  posterior  statement,  that  is, 

P(X/Y)  =  KL(X)dX  (3-1) 

being  only  variant  with  the  likelihood  function. 


3.1  BAND-LIMITED  WHITE  GAUSSIAN  NOISE 


Let  us  examine  the  band-limited,  white  Gaussian  noise  case  having  zero 
mean.  As  per  equation  (3-1)  for  equi-probable  a  priori  probability,  the 
posterior  statement  for  signal  detection  becomes 


t 


P(X/Y)  =  K  exp 


(Y-X)2dt 


dX 


(3-2) 
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Application  of  maximum  likelihood  method  to  equation  (3-2)  involves  min¬ 
imization  of  the  integral  term  therein.  Minimization  of  this  integral 
term  is  in  itself  a  statement  of  the  least  squares  method.  For  the 
Gaussian  case,  maximum  likelihood  always  encompasses  least  squares.* 
Application  of  least  squares  to  a  general  time  function  X  represented  as 
an  expansion  in  time  results  in  a  solution  in  terms  of  matrices  having  in¬ 
finite  rows  and  columns.  Approximations  to  X,  however,  permit  easy 
solution.  This  will  now  be  shown. 

Suppose  the  interval  (O,  T)  is  so  chosen  that  X  over  this  interval  can  be 
reasonably  approximated  as  X.  Equation  (3-2)  can  then  be  written  as 

P(X/ Y)  =  K  exp  j-i  f  (Y-X)2dt 

I  0J0 

where  X  may  be  expressed  as 

X(0,  T)  =  A  +  A.t  +  A,t2  +  •  •  •  +  A  tm  (3-4) 

o  i  b  m 

with  m  less  than  infinity.  ** 

It  can  be  shown  that  mininnisat  ion  of  the  integral  term  in  equation  (3-3)  is 
realized  by  choosing  the  A' s  in  accordance  to  the  matrix  equation 

[C}[A]  =  [B]  (3-5) 

whose  elements  are  given  by 

X  X 

C‘i=V4,i  VAo,Al.V"'Am:h*J0**  <3-6> 

or 


dX  (3-3) 


*  Reference  7  (Hald),  Chapter  8. 

**  We  could  just  as  well  choose  the  Fourier  series  expansion. 
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where  the  elements  of  the  rows  in  the  right-«ide  matrix  of  equation  (3-9) 
are  generated  from 


Sum  of 
Row  Elements 


1  _  /  »i+l  ml  ^(m-i+l) 

sj  “  <->  (m--iTlTT  T 


(3-10) 


Thus,  with  m  and  T  specified,  the  A's  can  always  be  determined  in  terms 
of  the  successive  integrals  of  Y  over  interval  (O,  T)  through  equation  (3-5). 
To  illustrate  this,  we  will  now  apply  least  squares  to  the  three  simplest 
cases,  that  is,  m=o,  m  =  1 ,  and  m=2. 


3.  2  SPECIFIC  EXAMPLES 

If  m=o  is  chosen,  then  the  approximation  to  X  is 
X(O.T)  =  X  =  Aq 
and  equation  (3-5)  becomes 

m  -  K»] 

or  Aq  is  given  by 


If  m=l  is  chosen,  then  the  approximation  to  X  is 
X(0,  T)  =  X  =  Aq  +  Ajt 
and  equation  (3-5)  becomes 


T^  "I 

T  h 

A 

o 

1  0 

rT  n 

rdt 

T2  t2 

N. 

T  -1 

(3-11) 


(3-12) 


(3-13) 


(3-14) 


(3-15) 
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or  Aq  and  Aj  are  given  through 


- 

—  — 

“  - 

A 

o 

2  6 
"T  t2 

fTYdt 

Jo 

Ai 

»  j 

6  12 

“2  — 5 
T 

If  m=2  is  chosen,  then  the  approximation  to  X  is 
X(0,  T)  =  X  =  Aq  +  Ajt  +  A2t2 
and  equation  (3-5)  becomes 


2  m3 

“  “ 

-T 

T  T • 

A 

o 

loo 

(  Ydt 
'o 

?  3  4 

rp  ^  * 

—  —  — 

Ai 

= 

T  -1  o 

rji  3  rji5 

TTT 

> 

L 

T2  -2T  2 

or  Aq,  A^,  and  A^  are  given  through 


A 

o 

3  24  60 

T  '2  _3 

fvd, 

Ai 

T  T 

_24  168  360 

""^2  rj,3  _,p4 

Jo 

f\*-2 

A2 

30  180  ^ 

p  ‘^r  e 
»  * 

.ai-*: 

(3-16) 


(3-17) 


(3-18) 


(3-19) 
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3.  3  COMMENTS  ON  IMPLEMENTATION 

Note  again  that  the  preprocessor  operations  involve  only  integrals  of  Y. 
Specification  of  the  approximation  to  X  will  determine  the  number  of  inte¬ 
grals  to  be  formed  by  the  preprocessor.  Determination  of  the  A's  in  the 
X  approximation  through  maximum  likelihood  is  not  a  preprocessor  oper¬ 
ation,  however,  but  a  part  of  the  subsequent  signal  processing  system. 

The  same  preprocessor  integral  measures  could  also  be  used  to  determine 
the  A's  by  a  method  other  than  maximum  likelihood. 

It  should  also  be  noted  the  simplicity  with  which  maximum  likelihood  can 
be  realized  in  a  subsequent  signal  processor  through  these  preprocessor 
integral  measures.  With  m  and  T  specified,  the  elements  in  the  matrix 
relating  the  A's  to  the  integral  measures,  as  in  equations  (3-13),  (3- 1 6) 
and  (3-19),  could  be  precalculated  and  stored  therein.  A  simple  set  of 
multiplication-addition  operations  each  observation  interval  would  then 
realize  maximum  likelihood. 

In  the  case  of  earth  transmittal  of  signal  data  from  a  spacecraft,  only  the 
integral  measures  need  be  transmitted.  Determination  of  the  A's  there¬ 
from  (or  implementation  of  signal  processor)  could  be  done  at  the  earth 
reception  station. 

Note  also  that  the  work  given  herein  is  applicable  to  least-squares  curve¬ 
fitting  in  general.  The  quantity  here  would  be  some  continuous  function  in 
time  which  can  be  approximated  by  some  truncated  power  series  in  time. 

The  constants  of  this  truncated  power  series  could  then  be  realized  through 
the  same  equation  (3-5)  matrix  relation  in  terms  of  the  integral  measures. 
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4.0  APPROXIMATE  MODEL  FOR  TIME- VARIANT  STOCHASTIC 

SIGNAL 


As  heretofore  implied,  the  time-variant  stochastic  signal  we  seek  to 
detect  in  the  presence  of  noise  can  often  be  approximated  over  any  one 
observation  interval  (O,  T).  As  in  the  previous  section  for  maximum 
likelihood,  that  approximation  consistent  with  information  measures  pro¬ 
vided  by  a  preprocessor  would  be  a  truncated  series  expansion  in  time. 

Choosing  to  approximate  the  signal  sought  by  some  truncated  time  series 
X,  the  posterior  statement  for  signal  detection  can  be  written  in  approxi¬ 
mate  form  as 

P(X/Y)  =  KP(X )  L(X)  (4-1). 

Of  primary  interest  is  that  approximation  of  X  which  assumes  X  to  be 
essentially  constant  over  any  one  observation  interval.  The  simplicity  in 
implementing  the  requirements  for  a  preprocessor  resulting  from  such 
an  approximate  signal  form  is  its  major  attribute. 

As  will  be  seen  herein,  it  is  most  convenient  to  consider  the  constant  X 
signal  over  any  one  observation  interval  to  represent  the  time-average  of 
X  over  this  interval;  that  is,  from  equation  (4-1),  the  posterior  statement 
for  signal  detection  for  any  one  observation  interval  becomes 

P(X/Y)  ~  KP(X)L(X)  ^KPfX^LfX*)  (4-2) 

with  If*  representing  the  time-average  of  X  over  an  observation  interval. 

The  latter  form  of  the  equation  (4-2)  posterior  probability  leads  to  a 
simpler  statement  of  the  general  Gaussian  noise  case.  This  we  shall  pro¬ 
ceed  to  show.  In  addition,  it  will  be  shown  that  the  time-average  esti¬ 
mate  can  be  interpreted  as  an  estimate  of  the  signal  at  the  mid-point  of 
an  observation  interval.  With  such  signal  mid-point  estimates,  it  is  then 
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possible  to  employ  a  variety  of  interpolation  schemes  in  a  subsequent 
signal  processing  system. 


4.  1  TIME-AVERAGE  ESTIMATE  FOR  BAND-LIMITED  WHITE 
GAUSSIAN  NOISE 

We  shall  now  develop  the  approximate  posterior  statement  for  signal  de¬ 
tection  in  the  presence  of  band-limited,  white  Gaussian  noise  with  zero 
mean  in  terms  of  the  time-average  estimates. 

The  density  function  of  the  time-average  of  noise  over  interval  (O,  T)  will 
then  be  Gaussian  in  form,  since 

(4-3) 

is  but  a  sum  of  terms  each  of  which  is  Gaussian.  Since  any  one  sample 
of  noise  is  assumed  to  have  zero  mean,  the  aforesaid  sum  will  also  have 
zero  mean.  Thus,  the  time-average  noise  density  distribution  will  be  of 
the  form 


P(K')  =  K  } 

2V 

2 

where  only  the  variance  a  must  be  determined. 

W 


(4-4) 


A  direct  way  of  obtaining  the  variance  of  equation  (4-4)  is  through  use  of 
the  basic  sampling  theorem.  Choosing  to  represent  the  noise  function  N 
by  a  finite  set  of  samples  taken  at  1/2W  equally  spaced  time  points  across 
(O,  T),  where  W  is  the  bandwidth  of  the  noise  spectrum,  the  time-average 
of  noise  appearing  in  equation  (4-3)  may  be  written  as 


(4-5) 
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where  Ny  is  any  one  noise  sample  of  the  finite  set.  Assuming  independent 
samples,  the  variance  of  this  sum  can  be  determined  as 


W 


=  (■ 


1 

■TWT  ’ 


2WT 


=  (2WT)22WT(NoW) 


N 

o 

2T 


(4-6) 


where  Nq  W  is  the  variance  of  any  one  noise  sample  and  Nq  the  mean 
noise  power  per  unit  bandwidth.  Thus,  the  density  function  of  equation 
(4-4)  for  the  time-average  of  band-limited,  white  Gaussian  noise  becomes 


p^*)  =  K  exp 


(tf)2 

n“7T 

o 


(4-7) 


Since  the  time-average  of  noise  is  by  definition 

^t  A  Yt  .  yt  (4_8) 

where 

xt  4  1 

X  -TJf 


hit 

Jo 


and  Y*  4  1  /Ydt 

Jo 


(4-9) 


then  use  of  equation  (4-7)  in  equation  (4-2)  gives  the  approximate  posterior 
statement  for  signal  detection  in  presence  of  this  noise  aB 


P(X/Y)  =  K  P(Xl)  exp  J  (4.10) 

As  in  previous  work,  we  choose  to  simplify  this  expression.  We  may  do 
this  by  examining  the  exponent  term  more  closely.  By  expansion,  this 
expression  becomes 


(4-11) 
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Since  the  integral  of  Y  in  practice  will  be  given,  the  first  term  on  the 
right-hand  side  of  equation  (4-11)  may  be  absorbed  into  the  constant  K, 
thereby  reducing  equation  (4-10)  to 

F(X/Y)  «  KP^)  exp  f  I 

H^LHH  ,4-i2) 

For  convenience  equation  (4-12)  is  written  as 

P(X/Y)  =  KP(X4)  exp  j-  ^jrj  exp  JVdtJ  (4-13) 

that  is,  we  desire  an  estimate  of  the  time-average  of  X  over  some  obser¬ 
vation  interval  (O,  T).  Note  that  our  results  coincide,  as  they  should, 
with  the  results  for  the  constant  signal  case  as  given  in  section  2.  0, 
equation  (2-10).  Identical  to  the  constant  signal  case,  therefore,  the  suf¬ 
ficiency  conditions  for  a  preprocessor  remain  as  the  integral  of  the  signal- 
noise  quantity  Y  over  interval  (O,  T).  The  simplicity  of  this  implementa¬ 
tion  again  is  our  justification  for  the  assumed  approximate  form  for  signal 
X. 


4.  2  TIME-AVERAGE  ESTIMATE  FOR  THE  GENERAL  GAUSSIAN 
NOISE  CASE 


The  approximate  posterior  statement  for  the  general  Gaussian  noise  case 
in  terms  of  the  time-average  estimates  can  be  directly  expressed  as 


P(X/Y)  =  KPQ?)  exp 


0?  +mn  )‘ 


exp 


(t  +MN) 


(4-14) 


where  is  the  mean  value  of  noise  (the  mean  of  any  one  noise  sample 
being  equal  to  the  time-averaged  mean  of  noise)  and  rr^.t  is  again  the 
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variance  of  the  time -averaged  noiae  quantity.  This  variance  is  obtain¬ 
able  through 

■>  Zo  z  f  > 

.4*—* «-i5> 

Too 

2 

where  cr^  is  the  variance  of  any  one  noise  sample  and  t)  is  the  noise 
correlation  coefficient.  Derivation  of  this  expression  for  variance  is 
shown  for  an  arbitrary  signal  in  Appendix  D  of  this  report. 

Again  note  that  the  requirements  for  a  preprocessor  remain  as  the 
integral  of  the  signal-noise  quantity  Y  over  some  observation  interval  (O,  T). 

As  a  side  interest,  equation  (4-14)  is  used  to  re-derive  the  approximate 
posterior  statement  for  band-limited,  white  Gaussian  noise.  This  is 
done  in  Appendix  E  of  this  report. 

The  approximate  posterior  statement  of  equation  (4-14)  is  also  obtainable 
directly  from  the  general  solution  for  correlated  noise  outlined  in 
section  2.9  by  equations  (2-30)  and  (2-31).  This  is  not  shown  herein, 
however. 


4.  3  TIME-AVERAGE  ESTIMATE  FOR  PRE -DETECTION  FILTERING 

The  heretofore  developed  posterior  statement  for  time -average  quantities 
is  now  extended  to  encompass  detection  problems  involving  a  restricted 
class  of  pre -detection  filters. 

As  stated  in  section  2.  3,  we  constrain  ourselves  to  filters  whose  output 
is  given  by 

XR  »  XR(t)  (4-16) 
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Some  difficulty  exists  if  we  attempt  to  develop  the  posterior  statement 
for  such  filter  action  directly  as  done  in  the  previous  section  for  R(t) 
equal  to  unity.  This  difficulty  exists  for  R(t)  represented  by  a  Fburier 
series,  that  is  , 


(4-17) 


(4-18) 

Replacement  of  3?*  in  the  equation  (4-8)  time-average  noise  definition  by 
the  corresponding  time-average  of  XR  sees  this  latter  quantity  to  be  zero 
for  observation  interval  equal  to  multiples  of  carrier  period  (since  X 
itself  is  assumed  constant  over  the  interval  of  interest).  Disappearance 
of  this  term  causes  equal  disappearance  of  X  from  the  eventual  posterior 
statement.  Proceeding  in  this  fashion,  therefore,  causes  the  time -average 
approach  to  break  down. 

Such  difficulties  can  be  avoided  by  working  with  (N*)2  directly  rather  than 
with  itself.  That  is,  by  writing 

- 1 

I2  2 

Fn  cos  t  +  en)J  -  ajjt  (4-19) 

— t  - 1 

00 

(tf)2  *  Y2  -2XY  2^  Fn  cos  (^j£t  +  en) 

- 1 

+  ^n  COB  ‘  +  (4-20> 


(rf)S 


R(t)  =  £  F  COS  (3}rt  +  6 1  ) 
n=0 


leading  to 

X„  = 
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Since  we  are  interested  in  these  time-averages  over  some  interval  (O,  T), 
equation  (4-20)  may  be  written  as 


(4-21) 


Moreover,  since  X  is  assumed  constant  over  the  interval  of  interest, 
where  the  constant  value  represents  the  time-average  of  X  over  the 
interval,  equation  (4-21)  may  be  written  as 


Y2dt  -  2* 


zX*  j  Yy 

~Tl  U  » 


cos  (2^n  t  +  ©n)dt 


+ 


dt  -  Ojft 


(4-22) 


(The  ambiguous  use  of  above  can  be  somewhat  misleading,  though  here 
it  is  meant  to  represent  some  constant  value  which  reflects  time-average 
of  X  over  the  interval.  ) 


Insertion  of  equation  (4-22)  into  the  approximate  posterior  statement  for 
time-average  estimate  of  the  equation  (4-10)  type  results  in 


P(X/ Y)*KP(X*)  exp  j  i - L-  f  Y2dt  |  exp  j- 

»  2Ta^t  Jo  *  )  2T<r£t 

,n']2dl|- 


Fn  cos(— ^  t  +  ®n)dt 


(4-23) 
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With  Y  being  given,  the  first  exponent  term  can  be  absorbed  into  K, 
resulting  in 


P(X/Y)  =  KP(X*)  exp 

lx4 

exp  - T 


2 


dt 


(4-24) 


This  is  a  satisfactory  posterior  statement.  (For  simplicity  sake,  the 
noise  mean  value  was  assumed  zero.  ) 

Equation  (4-24)  calls  for  preprocessor  requirements  to  be  an  infinite  set 
of  filters  matched  to  the  sinusoids  of  the  harmonics  representing  the  R(t) 
Fourier  expansion.  (In  keeping  with  Bection  2.  3,  it  was  again  assumed 
that  time  behavior  of  R(t)  is  fully  known.  ) 


4.  4  MID -POINT  ESTIMATE 

Much  as  the  discussion  in  section  3.  0  on  maximum  likelihood,  the  sub- 
!  sequent  work  also  relates  to  signal  processing  which  might  be  done  with 
those  measures  provided  by  the  detection  system  preprocessor.  It  is 
given  as  added  support  to  the  time-average  interpretation  utilized  hereto¬ 
fore  in  sections  4.  2  and  4.  3. 

Primary  value  in  such  interpretation  is  that  signal  time-average  over 
observation  interval  best  reflects  signal  value  at  mid-point  of  observation 
interval.  With  such  point  estimates  of  signal  available,  various  inter¬ 
polation  schemes  are  possible  in  a  signal  processing  operation. 

To  show  this,  we  draw  from  Appendix  F  the  expression  describing 
variance  of  difference  between  signal  value  at  any  time  during  an 
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observation  interval  (O,  T)  and  time -average  of  signal  over  this  interval. 
The  equation  for  this  variance  is 

<’x-X'  ■  4  | 1  •  Tj[  px" 

where  O^(t)  is  the  signal  correlation  coefficient. 

As  expected,  the  variance  of  equation  (4-25)  varies  with  time  over  the 
interval,  depending  upon  what  point  in  time  is  chosen  for  the  single 
signal  value. 

From  the  nature  of  the  expression,  we  wOuld  expect  the  variance  to  have 
a  minimum  at  some  point  in  the  interval,  with  intuition  suggesting  the 
mid-point  as  the  most  likely  place. 

Referring  to  the  derivation  in  Appendix  F,  we  see  the  signal  correlation 
coefficient  used  therein  such  that  it  is  always  either  zero  or  positive 
valued,  that  is, 

0Spx(t)si  •  (4-26) 

Thus,  the  integrals  appearing  in  equation  (4-25)  will  always  be  positive, 
monotonically  increasing  functions.  This  being  so,  the  variance  will  then 
be  least  when  the  sum  of  the  first  two  integrals  is  a  maximum. 

Owing  to  the  condition  in  equation  (4-26)  and  that  in  practical  situations 
P^(t)  will  itself  be  monotonically  decreasing  over  a  prescribed  interval 
(O,  T),  the  sum  of  these  integrals  will  be  maximum  when  their  upper 
limits  are  both  one  half  the  interval  length  (T/2).  Thus,  the  minimum 
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variance  of  the  difference  between  a  signal  value  over  the  interval  and  the 
signal  time-average  over  this  interval  occurs  at  the  mid-point,  and  is 


(4-27) 


This  we  see  is  in  agreement  with  our  intuition  heretofore  stated.  As  an 
aside,  the  maximum  variance  occurs  at  the  end  points  of  interval  (O,  T), 
being 


2 

■  T 


(4-28) 


The  most  important  consequence  of  equation  (4-27)  is  that,  if  the  pre¬ 
processor  is  constrained  to  only  the  integral  of  the  signal-noise  quantity, 
then  utility  of  this  preprocessor  measure  in  a  subsequent  signal  processor 
is  maximized  when  this  single -integral  measure  is  interpreted  as  an 
estimate  of  signal  value  at  mid -point  of  observation  interval. 


Assuming  such  an  interpretation  in  a  signal  processor  then  permits  a 
variety  of  interpolation  schemes  to  be  employed  therein.  If  boxcar 
interpolation  were  used,  the  result  would  be  the  step-like  form  (a  form 
which  itself  could  have  been  taken  as  the  starting  point  for  an  alternate 
derivation  of  the  approximate  posterior  statement). 

Linear  interpolation  between  these  mid-point  estimates  could  also  be  used 
to  better  represent  signal  time  form,  as  well  as  some  more  sophisticated 
scheme  predicated  on  the  more  ideal  sin  wt/t  type  interpolation. 


Perhaps  it  is  important  here  to  point  out  that,  much  like  the  preprocessor 
has  the  task  of  maximizing  relevant  information  from  signal  plus  noise, 
so  must  the  subsequent  signal  processor  see  that  preprocessor  output  is 
most  effectively  utilised. 
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As  further  point,  let  us  develop  the  posterior  statement  for  mid-point 
estimate  in  terms  of  time-average  quantities.  We  may  do  this  by  commenc¬ 
ing  with  the  joint  probability  statement  involving  mid -point  ,  signal 

time-average  Y*  and  signal -noise  time -average  Y*.  First  writing  the 
identities 


P(XM,  Y*.  Y*)  =  P^.Y*)  P(Yt/xMYt) 
=  p(xM,  ySp^/Y4) 

=  p(xM,  Y*)  MY*) 

and 

P(XM,Yt,Yt)  =  P(Yt)P(XM,  YVY*) 

where 

P(Yt/xMYt)  =  p(Y  VY4)  =  MY*) 


we  may  then  form 


-  P(xM.xl)L(xt) 

P(XM^  =  — ^ - 


=  K  P(XM,Yt)  L 


(4-29) 


(4-30) 


(4-31) 


(4-32) 


where  P(Y*),  with  Y*  given,  is  absorbed  into  K. 


The  desired  posterior  statement  then  is  a  marginal  probability  of  equation 
(4-32).  This  marginal  probability  may  be  formed  by  summing  over  all 
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a  .  Denoting  an  arbitrary  value  of  3J*  by  X*.  and  summing  over  all  i, 
the  desired  posterior  statement  then  becomes 

PtXj^/T4)  =  £  P(xM> 

=  K £  P(XM,  Jfy  L(Y‘)  (4-33) 

Compared  with  equation  (4-2),  we  see  only  the  a  priori  probability  for 
signal  time -average  being  replaced  by  a  joint  probability  and  a  request 
made  for  a  summation  over  all  possible  signal  time -averages. 

As  an  oft  repeated  reminder,  we  again  say  that  any  manipulations  involving 
interpretation,  interpolation  and  the  like  in  a  subsequent  signal  processor 
in  no  way  affect  requirements  for  detection  system  preprocessor. 
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5.0 


A  PRIORI  CONSIDERATIONS  FOR  APPROXIMATE  MODEL 


To  consider  a  detection  system  for  any  application,  its  effectiveness  ulti¬ 
mately  must  be  demonstrated.  Since  that  employed  will  invariably  repre¬ 
sent  an  approximation  to  the  ideal  set  of  requirements,  as  implied  through 
the  series  solutions  of  section  2.  0,  it  must  be  shown  to  be  the  best  com¬ 
promise  between  performance  required  and  allowable  equipment  costs. 

The  design  should  reflect  maximum  "performance-per-equipment-cost" 
figures  Of  merit. 

One  detection  scheme  can  be  weighed  against  another  through  the  simple 
expedient  of  comparing  likelihood  functions,  with  the  means  and  variances 
thereof  receiving  attention.  Mean  value  will  indicate  sensitivity  of  likeli¬ 
hood  function  to  signal  fluctuation,  and  variance  the  deviation  about  the 
mean.  For  the  Gaussian  type  likelihood  functions  used  in  our  previous 
work,  consideration  of  these  two  parameters  should  suffice  for  such  com¬ 
parison. 

Ultimately,  however,  comparison  of  likelihood  functions  must  be  supple¬ 
mented  with  an  evaluation  of  total  system  behavior,  signal  processor  in¬ 
cluded.  This  is  necessary  to  avoid  heavy  emphasis  on  consequential 
changes  in  likelihood  function  which  have  little  effect  on  the  a  posteriori 
probability. 

To  do  this  indeed  requires  knowledge  of  the  a  priori  distribution,  some¬ 
thing  which  may  not  be  available.  If  not  known,  a  best  guess  must  be  made 
regarding  this  distribution  and  a  hypothetical  model  set  forth,  at  least  for 
analysis  purposes.  The  model  need  not  necessarily  be  the  same  a  priori 
distribution  underlying  the  ultimate  signal  processor  functions,  however. 

Subsequent  work  will  assume  the  a  priori  distribution  to  be  Gaussian, 
meaning  the  a  posteriori  probability  itself  will  also  be  Gaussian.  The 
mean  and  variance  of  the  posterior  statement  will  receive  primary  atten¬ 
tion.  Here  the  mean  will  be  shown  to  reflect  sensitivity  of  the  detection 
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system  to  signal  fluctuation,  with  variance  describing  behavior  about  the 
mean. 


The  approximate  posterior  statement  based  on  time -average  quantities 
developed  in  section  4.  0  will  be  used  in  subsequent  work.  Interpretation 
of  performance  obtained  therethrough  will  be  presented.  For  reference 
sake,  this  expression  is  repeated  below: 


P(X/Y)  =  KP<X4)  exp 


t  ^ 
2atf 


(X*  +  4n) 
- 1 - 


(5-1) 


2 

where  is  the  mean  value  of  noise  and  Cj^t  the  variance  of  the  time- 
average  of  noise  given  by 


PN(t)dt2 


(5-2) 


2 

with  ctn  the  variance  of  any  one  noise  sample  and  p^(t)  the  noiBe  correla¬ 
tion  coefficient. 


5.  1  GAUSSIAN  A  PRIORI  DISTRIBUTIONS 

We  now  set  down  several  forms  for  the  a  priori  distribution  vhich  will  re¬ 
ceive  attention  in  the  subsequent  sections. 

Where  behavior  over  any  one  observation  interval  (O,  T)  adequately  de¬ 
scribes  the  signal  stochastic  process,  the  a  priori  Gaussian  probability 
can  be  written  in  terms  of  its  density  function  as 

P(X*)  =  1  y  exp  I  -  dX*  (5-3) 

a£t  *  2a|t  > 
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where  i»  the  average  value  of  the  aignal  and 


p  (t)dt2 
x 


(5-4) 


2 

with  the  variance  of  the  signal  and  px(t)  the  signal  correlation 
coefficient. 


Where  behavior  over  any  one  observation  interval  is  also  dependent 
upon  behavior  over  previous  interval,  the  second  distribution  for 
the  Gaussian  a  priori  probability  applies,  which  becomes 


P(X 


n. 


X  .)  = 

n-1 


2  It  a 


exp 


I - 5 - 

'  2<r~t  (1-p-t) 

X  X 


[<<  - »/-  2»:1  '<  -“,1  K.rV 


(5-5) 


where  represents  time-average  over  present  interval, 
time -average  over  previous  interval,  and  p- 1  is  defined  by 
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(5-6) 


o-t  = 
x 


’£lT'.“‘  If'-- 


■T 


TT 


l/p*(t,dt 


The  derivation  of  p^t  may  be  found  in  Appendix  H. 


If  the  time-average  is  interpreted  as  the  mid-point  value  of  the 
signal,  as  per  section  4,  4,  and  if  behavior  over  any  one  interval 
adequately  describes  the  signal  process,  the  a  priori  probability  is 
now  replaced  by  a  joint  probability  in  the  mid-point  value  X^  and 
the  time-average  signal  value  X*.  This  is  shown  in  equation  (4-33) 
of  section  4.4.  For  a  Guassian  X,  thus  joint  probability  is 


P<XM,*V 


Zita 


o 

X  x  l 


exp 


XM,X 


1 


P“  -t 

2  .  j.  /v  V  /vt 


2avcr-t  (l-p  -t) 

XM,X 


x  x 


x*  <XM^x><X  -"x> 

M, 


+  «•-(./] 


dXMdXl 


where  p  -t  is  defined  by 
XM,  x 


2a. 


P  -t 

XM,  x 


2  Tr  T/2 

T-  iPj'<t|dl  Vo 


a  a-t 

X  X 


p„(t)dt‘ 


o-'o 


(5-7) 


(5-8) 


The  derivation  of  p  -t  is  presented  in  Appendix  I. 

XM,  x 
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GAUSSIAN  A  POSTERIORI  DISTRIBUTIONS 


We  now  present  the  a  posteriori  probabilities  based  on  the  time-average 
likelihood  function  of  equation  (5-1)  for  the  three  a  priori  distributions 
presented  in  section  5. 1. 
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For  the  time-average  Gaussian  a  priori  distribution  for  any  one  observa¬ 
tion  interval  given  in  equation  (5-3),  the  posterior  probability  becomes 


P(X/Y)  - - —  exp 


(X4  -M  x)2 


2a, 


dX 


where 


>  (5-9) 


and 


Where  present  signal  behavior  is  conditioned  on  behavior  during  previous 
observation  interval  as  described  through  the  joint  statement  of  equation 
(5-5),  the  posterior  statement  can  be  written  as 


P«n  *»-lY>;rKP«»  *Ll>  "P 


f*n  +  *V  }VM 

exp  J - ? -  J  Ydt 
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rat 


2a 


n 


V  (5-10) 
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where 
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off  ( 1  -P^)MX+  0^ tXL  j  +  oh  (1  -p|t) (^  /0Ydt-<i^ 


o|t+<4t(l-4> 


_  2  _  ^  (1_4t) 

J2  2  2  7!  2T 

+  a^  (1"  p^t) 


(5-10) 


If  the  time-average  of  the  signal  is  interpreted  as  the  mid-point  value,  : 

then  as  per  equation  (4-33)  in  section  4.  4  the  posterior  statement  becomes 

\Z  \ 


P<XMlrt  =  KEP(XM,^1  “P 


4  V 

I  2 
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— — f  Ydt 
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Upon  employing  equation  (5-7),  we  then  have 


<xM|Yt)'ST  exp 
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where 
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^Xm.^  W(T  /oYdt 


and 


a3  = 


ffx¥  [W*1  -4M,xt)+0j|] 


(5-11) 


(5-12) 


5.  3  PERFORMANCE  FOR  TIME -AVERAGE  A  PRIORI  DISTRIBUTION 

Let  us  now  examine  performance  of  a  system  employing  the  step-like 
approximation  of  a  signal  quantity.  We  choose  first  the  posterior  state¬ 
ment  of  equation  (5  -9),  which  is  based  on  a  Gaussian  a  priori  distribution 
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for  any  one  observation  interval.  Examination  thereof  of  information 
obtained  through  detection  can  be  made  by  utilizing  the  "information  change" 
definition  developed  in  Appendix  G.  Therein  information  change  was 
defined  as  a  fractional  change  in  variance,  that  is, 


IC 


A  (prior  Variance]  -  [posterior  Variance] 


T 


Prior  Varianc 


F 


Thus,  using  equation  (5-9)  the  above  equation  becomes 


(5-13) 


<?Trt 

IC.  =  - - -  (5-14) 

1  2  2 

V  + 

From  equation  (5-14)  we  see  change  in  information  through  detection  is 
zero  in  the  limit  where  noise  becomes  infinite,  Examination  of  the  time- 
average  variances  also  sees  this  information  change  become  a  constant 
less  than  unity  as  length  of  observation  interval  becomes  infinite. 


In  practical  situations,  the  information  change  definition  should  be 
satisfactory.  The  formal  definitions  of  information  gain  (See  Appendix  G) 
should  also  be  applicable  with  no  foreseeable  difficulty.  The  simple  in¬ 
formation  change  should  usually  suffice  for  design  purposes,  though  the 
formal  definitions  should  be  used  for  extensive  information  flow  studies 
regarding  the  overall  system. 

Of  great  interest  also  is  an  understanding  of  how  well  the  detection  system 
follows  signal  behavior.  One  way  of  examining  this  is  to  assume  the 
system  proper  will  choose  the  most  probable  value  of  posterior  statement 
as  the  signal  value  during  the  observation  interval.  For  the  Gaussian 
posterior  statements,  this  will  be  the  average  value  thereof. 

Now,  signal  will  fluctuate  during  the  observation  interval  with  a  standard 
deviation  cr^,  whereas  the  time-average  detected  quantity  gradually 
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becomes  equal  to  the  sum  of  signal  and  noise  means,  as  length  of  obser¬ 
vation  goes  to  infinity.  Thus,  in  the  limit,  the  time -average  detected 
quantity  is  insensitive  to  signal  fluctuations  (or  noise  fluctuations,  for 
that  matter). 

There  are  many  ways  in  which  sensitivity  of  detected  quantity  could  be 
expressed.  Perhaps  one  of  the  simplest,  and  still  satisfactory  methods  is 
to  define  a  sensitivity  factor  based  on  ratio  of  change  in  mean  of  detected 
quantity  for  change  in  signal  time -average  to  standard  deviation  of  signal, 
that  is, 


dX4 


or  for  the  posterior  statement  of  equation  (5-9), 


.  -L 

CTx 


4* 


& + 


dX4 

°X 


(5-15) 


(5-16) 


To  simplify  matters,  equation  (5-16)  can  be  examined  at  one  standard 
deviation  of  X4,  giving 


°X. 


(5-17) 


This  sensitivity  factor  then  gives  us  a  direct  feeling  for  how  well  time- 
average  detected  quantity  responds  to  signal  behavior  during  an  observa¬ 
tion  interval  (Note  that  as  T  approaches  infinity  goes  to  zero).  Use  of 
both  the  equation  (5-15)  sensitivity  factor  and  the  equation  (5-13)  infor¬ 
mation  change  can  act  as  "rule-of-thumb"  guides  for  the  system  designer. 


5-8 


5.4 


PERFORMANCE  FOR  CONDITIONAL  TIME -AVERAGE  A  PRIORI 
DISTRIBUTION 


Use  of  the  conditional  time-average  a  priori  distribution  to  develop  the 
posterior  statement  is  now  examined.  The  posterior  statement  in  question 
is  given  by  equation  (5-10). 

Using  the  equation  (5-13)  definition  of  information  change,  we  have 


2  2  .  2  2 
a^t(i  -  Pg t)  +  Pgt  o^t 

o^t'  +  ffyt  (1  -  Py t) 


(5-18) 


Here,  it  is  seen  that  the  change  in  information  is  heavily  predicated  on 
correlation  in  the  signal  quantity,  since  knowledge  of  time-average  of 
signal  during  previous  interval  is  presumed  known. 


A  question  might  arise  regarding  where  this  knowledge  of  previous 
signal  behavior  is  obtained.  Generally  such  knowledge  will  not  exist. 
That  estimate  of  signal  time-average  determined  during  previous 
observation  interval  could  be  employed.  Though  in  error,  no  serious 
difficulty  can  be  envisioned  in  its  use  in  a  properly  designed  system. 


The  sensitivity  factor  of  equation  (5-15),  evaluated  at  one  standard  deviation 
of  signal  time-average,  for  this  posterior  statement  becomes 


c^t  (1  -  P-^t) 

^  +  °Xt(1 


(5-19) 


Notice  as  the  correlation  coefficient  approaches  unity  the  sensitivity 
becomes  sero.  This  is  as  it  should  be  for  heavily  correlated  signals. 
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PERFORMANCE  FOR  MID-POINT  A  PRIORI  DISTRIBUTION 


5.  5 


The  posterior  statement  for  the  mid -point  interpretation  given  in  equation 
(5-12)  leads  to  the  following  expressions  for  information  change  and 
sensitivity: 


and 


°x  -  ***_ 

+  <Txff3ft 

°X  ‘ 

a 

X 

CTX*  +  o£t 

°X  +  "ft* 


(5-20) 


(5-21) 


These  expressions  are  not  examined  to  determine  actual  gain  realized 
through  such  interpretation.  This  can  be  done  by  comparing  the  foregoing 
quantities  with  equation  (5-14)  and  (5-17). 
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CONCLUDING  REMARKS 


6.  0 


The  a  posteriori  probability  approach,  as  developed  by  P.  M.  Woodward, 
was  shown  to  lead  directly  to  sufficiency  conditions  for  a  detection  system; 
reduction  of  these  conditions  to  practical  form  was  shown  to  be  possible 
by  signal  representation  through  series  expansion. 

Examination  thereof  for  cases  seeking  signals  with  unknown  parameters, 
especially  those  faced  by  a  passive  detection  system,  revealed  the  "nor¬ 
mally  assumed"  basic  requirements  for  a  detection  system  to  be  an  im¬ 
proper  statement  of  the  problem.  Representation  of  an  unknown  signal 
by  an  infinite  series  with  arbitrary  coefficients  clearly  demonstrates  this 
point.  Through  this  representation,  a  restatement  of  these  sufficiency 
conditions  showed  detection  system  requirements  to  be  the  formation  of  an 
infinite  set  of  statistical  estimates  reflecting  all  information  that  can  be 
obtained,  from  which  the  arbitrary  coefficients  can  be  formed.  Forma¬ 
tion  of  these  arbitrary  coefficients,  however,  was  shown  to  be  a  task  not 
of  the  detection  system,  but  a  task  of  some  subsequent  signal  processing 
system. 

Two  series  expansions  received  attention  in  this  report:  power  series  and 
Fourier  series  in  time.  The  various  sets  of  detection  system  require¬ 
ments  obtainable  through  these  two  series  expansions  alone  provide  the 
system  designer  with  rather  impressive  guides  by  which  to  resolve  a  wide 
variety  of  problems.  In  fact,  most  detection  system  problems.,  both  ac¬ 
tive  and  passive,  should  lend  themselves  to  solution  therewith. 

Solutions  through  other  series  expansions  are  also  possible.  In  a  practi¬ 
cal  sense,  however,  one  should  choose  those  forms  of  expansion  which 
converge  most  rapidly  for  the  types  of  signals  to  be  accommodated  and 
whose  solutions  thereby  lead  to  ready  implementation.  Rapid  convergence 
implies  only  a  few  terms  of  the  series  expansion  need  be  considered  to 
approximate  the  ideal  set  of  requirements. 
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The  solutions  given  in  this  report  revealed  present  practices  as  approxi¬ 
mations  thereof;  the  exact  nature  of  these  approximations  is  clearly  ex¬ 
pressed.  Moreover,  the  form  of  solution  given  allows  logical  improve¬ 
ments  to  present  practices. 

The  consequences  of  the  reported  work  with  regard  to  present  and  future 
activities  cannot  be  stated  at  the  present  moment;  this  work  must  stand 
the  test  of  time.  Moreover,  an  enormous  amount  of  labor  remains  to 
further  explore  this  work,  and  especially  to  reduce  it  to  practice. 
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APPENDIX  A 


REAL-TIME  COMPUTER  OPERATION 

As  background  to  the  work  presented  in  the  body  of  this  report,  we  exam¬ 
ine  a  facet  of  the  digital  computer  world  which  finds  itself  confronted  with 
the  basic  problem  of  suitable  signal  characterization.  Specifically,  we 
discuss  how  a  serial,  general-purpose  digital  computer  accommodates  a 
typical  real-time  problem,  and  roughly  relate  how  improved  performance 
is  obtainable  through  better  signal  representation,  better  representation 
through  function  expansions.  The  similarity  between  this  computer  pro¬ 
blem  and  signal  detection  is  very  striking  in  many  respects. 

First,  consider  the  typical  general-purpose  digital  computer.  The  com¬ 
puter  has  stored  in  its  memory  section  a  set  of  functions  |f|  which  ex¬ 
plicitly  describe  some  real-time  problem.  These  functions  vary  in  ac¬ 
cordance  to  a  set  of  variables,  js}  signal  set,  which  are  provided  by  the 
outside  world.  The  computer,  under  direction  of  a  stored  program  P, 
operates  sequentially  on  jF}  for  a  given  set  of  js}  values  to  generate  an¬ 
other  set  of  values  representing  desired  output  quantities  jo}  .  The  oper¬ 
ation  describing  provision  of  an  jo}  value  set  for  a  given  js}  value  set  is 
called  the  program  compute  cycle.  With  one  set  of  jo}  values  determined, 
the  compute  cycle  is  repeated  for  another  js}  value  set.  In  real-time  ap¬ 
plications,  a  time  sequence  of  such  repeated  operations  (or  compute  cycles) 
constitutes  a  "running"  solution  of  the  problem. 

The  time  length  of  compute  cycle  for  a  given  general-purpose  computer 
varies  with  problem  complexity,  as  well  as  composition  of  program  P 
dictated  by  a  particular  problem.  Significant  here  is  the  fact  that  a  fixed 
set  of  js}  values  is  used  for  each  compute  cycle  (that  is,  signals  from  the 
outside  world  are  assumed  constant  during  each  compute  interval).  Any 
signal  changes  over  the  interval  thus  will  not  reflect  in  the  computation, 
and  accordingly  the  generated  output  set  of  {0}  values  will  be  in  error. 
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When  these  errors  are  of  a  cumulative  nature,  extremely  serious  limita¬ 
tions  can  be  placed  on  the  spectrum  of  real-time  problems  suitable  to  such 
computers. 

In  aerospace  applications  especially,  present  state-of-the-art  operational 
equipment  places  rather  high  cost  factors  on  equipment  weight,  volume, 
power,  reliability  and  the  like.  Many  times  these  equipment  costs  are 
unacceptable,  and  the  digital  computer  is  omitted  from  further  considera¬ 
tion,  or  worse,  system  development  marks  time  until  equipment  state-of- 
the-art  becomes  more  favorable. 

The  foregoing  situation  will  always  constitute  an  upper  limit  on  the  spec¬ 
trum  of  real-time  problems  suitable  to  serial,  general-purpose  operation 
as  described  above.  As  the  state-of-the-art  improves,  the  limit  will  be 
raised. 

This  then  poses  a  question  regarding  the  possibility  of  raising  the  upper 
limit  with  equipment  state-of-the-art  remaining  constant.  If  equipment  is 
fixed  and  if  We  also  constrain  ourselves  to  serial,  general-purpose  opera¬ 
tion,  the  only  consequential  games  we  can  play  are  with  the  signal  set  |sj  . 
Fundamentally,  the  games  played  here  involve  a  study  in  signal  representa¬ 
tion,  or  better,  whether  the  normally  assumed  constant  |s|  values  for  a 
compute  interval  is  an  efficient  way  of  representing  signals.  Here,  we  see 
the  mapping  of  many  signal  members  (representing  signal  behavior  over 
the  time  interval)  into  one  member  of  another  set.  This  many-to-one  map¬ 
ping  is  an  irreversible,  information -losing  proposition.  So  long  as  the 
relevant  information  lost  is  negligible,  the  mapping  is  acceptable.  Other¬ 
wise,  we  must  either  seek  a  more  representable  member  for  the  many-tp- 
one  mapping  or  consider  a  many -to -many  mapping.  Desirably,  this  latter 
many-to-many  mapping  should  be  such  that  its  membership  for  any  one 
mapping  operation  is  controllable  and  its  members  readily  formed  (that  is, 
members  formable  at  minimum  equipment  costs).  Since  increase  in 
mapped  membership  will  reflect  decrease  in  relevant  information  lost  in 


A-2 


mapping,  such  control  of  membership  should  permit  minimum  complexity 
in  signal  representation  consistent  with  allowable  information  loss. 

Choosing  to  represent  any  function  expansion  of  each  over  an  arbitrary 
compute  interval  as  a  power  series  in  time,  it  is  then  seen  that  the  coeffi¬ 
cients  of  this  power  series  represent  a  new  mapped  multi -membe red  set. 
Its  membership  for  any  one  mapping  is  controlled  simply  by  specifying 
the  number  of  non-zero  coefficients.  These  power  series  could  represent 
an  expansion  through  an  orthogonal  function  set,  an  approximation  thereof, 
or  even  some  truncated  Taylor  series.  The  coefficients  themselves  might 
be  obtained  in  a  variety  of  ways.  They  might  be  determined  through  past 
differences  of  signal  samples  or  some  other  measure  of  past  signal  be¬ 
havior  (as  integral  measures,  for  example).  If  some  computation  delay  is 
permissible,  signal  behavior  over  present  interval  of  interest  could  be 
measured  to  determine  these  coefficients. 

Most  important  are  those  power  series  expansions  whose  coefficients  can 
be  determined  through  integral  measures  of  signal  behavior,  especially 
that  set  of  measures  representing  successive  integrals  of  signal  over  some 
time  interval.  This  set  can  be  readily,  and  cheaply  formed  in  a  digital 
computer  through  use  of  a  periodic,  high-speed  adder  (in  Librascope 
phraseology,  this  high-speed  adder  is  termed  the  Sigmator).  Moreover, 
this  set  can  be  readily  absorbed  into  the  problem  mathematical  structure. 
The  mathematics  describing  expansion  through  integral  measures  is  re¬ 
lated  in  Appendix  B. 

Such  expansions  have  been  shown  to  be  extremely  effective  in  certain  real¬ 
time  computer  problem  formulations,  especially  those  involving  integrals 
of  more  than  one  variable.  Orders  of  magnitude  improvement  are  realiz¬ 
able  without  altering  the  computer  structure.  Their  utility  was  first  intui¬ 
tively  seen  some  5-6  years  back  by  R.  R.  Williamson,  and  developed  since 
then.  The  present  writer  himself  probed  numerous  special  cases  involving 
such  expansions  during  essentially  the  same  time  span.  Each  probe  re¬ 
sulted  in  success.  R.  R.  Williamson  ultimately  wrote  a  report  titled 
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"Real-Time  Control  Computers"  (Reference  5)  in  early  1962,  wherein  he 
indicated  with  reasonable  mathematical  substance  formation  of  these  ex¬ 
pansions,  This  report  coupled  with  a  number  of  formal  and  informal  in¬ 
vestigative  reports  by  the  present  writer  served  as  the  foundation  for  a 
report  titled  "Application  of  GP-Sigmator  Hybrid  to  Real-Time  Problems" 
(Reference  6),  also  prepared  by  the  writer  of  this  report,  in  late  1962. 
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APPENDIX  B 


FUNCTION  EXPANSION  THROUGH  INTEGRAL  MEASURES 

Consider  a  variable  X  represented  by  an  expansion  over  time  interval 
(O,  T)  comprised  of  a  linear  set  of  orthonormal  functions,  that  is. 


X(0,  T)  =  FQy>o(t)  +  FjiP-jOi)  +  •  •  *  +  Fj  +  *  •  * 
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where  the  (t) 1  s  are  the  orthonormal  functions  and  the  F’s  constants  de¬ 
fined  by 


F  =  /  .(t)  X(t)dt 

J  Jn  J 


(B-2) 


Choosing  to  express  any  orthonormal  set  by  their  corresponding  power 
series  in  time,  the  jth  orthonormal  function  then  becomes 


J 


(B-3) 


Then  inserting  this  form  into  equation  (B-2)  gives 


F.  = 
J 


X(t)dt 


(B-4) 


Assuming  a  imiformly  convergent  series  in  interval  (O,  T),  we  may  make 
use  of  the  fact  that  the  integral  of  an  infinite  sum  is  equivalent  to  the  sum 
of  the  integrals.  Thus,  equation  (B-4)  can  be  written  as 


F.  = 
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tkX(t)dt 


(B-5) 


The  a^  terms  are  known  since  they  describe  those  orthonormal  functions 
we  happen  to  select.  Only  the  integral  terms  remain  to  be  determined. 
Direct  implementation  of  these  integrals  could  pose  a  serious  problem 
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since  multiplication  accompanies  the  integration.  This  multiplication  can 
be  circumvented,  however,  by  the  simple  expedient  of  expanding  the  inte¬ 
grals  through  integration  by  parts.  This  we  illustrate  below: 


=  Jf  X(t)dt  =  jf  X(t)dt 
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Etc. 

or  the  kth  term  can  be  expressed  as 
k+1 


(B-7) 


where  s  1[X(t)]|  represents  the  ith  integral  of  X(t)  over  interval  (O,  T). 


The  {f}  set  of  equation  (B-l)  can  then  be  expressed  in  terms  of  the  {f*} 


set  of  equation  (B-6)  through  the  matrix  equation 
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that  is, 
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Thus,  we  see  that  the  variable  X(t)  can  in  general  be  expressed  by  a  power 
series  in  time  of  the  form 


X(0,  T)  =  Aq  +  Ajt  +  A2t2  +••■•+  Aktk  +  •  •  • 
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where  the  A's  are  determined  through  integral  measures  of  X(t)  over  in¬ 
terval  (O,  T),  as  reflected  through  equations  (B-l),  (B-3),  (B-6),  and 
(B-9).  The  particular  relationship  between  the  A's  and  the  integral  meas¬ 
ures  depends  upon  the  function  expansion  chosen.  In  general  form,  how¬ 
ever,  the  A's  may  be  expressed  as 
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or,  since  the  F's  are  given  by  equation  (B-9),  the  A's  become 
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where  the 


are  defined  by  equation  (B-7). 
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As  stated  in  Appendix  A,  the  integral  terms  appearing  in  the  {f*}  set  are 
easily  formed  in  a  digital  computer  through  use  of  a  periodic,  high-speed 
adder  (Sigmator,  in  Librascope  language).  Since  such  an  adder  is  part  of 
the  normal  complement  of  a  real-time  digital  computer,  these  power 
series  expansions  can  be  used  to  excellent  advantage  on  real-time  prob¬ 
lems  to  enhance  computer  performance  at  essentially  zero  added  cost. 


In  a  digital  computer  and  also  in  our  detection  problem,  it  is  the  form  of 
the  expansion  which  is  important  —  not  the  specific  expansion  —  this  we 
can  choose  to  best  suit  the  problem.  Simply  stated,  the  power  series 
form  of  expansion  is  important  because  it  leads  to  simple  implementation. 
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In  our  work  on  signal  detection,  little  interest  is  shown  in  specific  power 
series  expansions.  We  primarily  deal  with  the  general  form  given  in 
equation  {B-10),  under  the  tacit  assumption  that  particular  A's  can  be 
determined  if  desired. 
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APPENDIX  C 


.AN  ARGUMENTIVE  RESOLUTION  OF  INTEGRAL  XY 

Relying  heavily  on  the  reasoning  developed  in  Appendix  A,  we  now  endeav- 
or  to  resolve  integral  XY  in  much  the  same  way.  To  do  this,  we  find  it 
instructive  to  commence  with  the  constant  signal  case  presented  in  section 
2.  1  and  given  by  equation  (2-10). 

If  signal  is  known  to  be  constant,  detection  system  need  only  form  the  in¬ 
tegral  of  Y.  This  integral  constitutes  an  irreversible  operation  by  the  de¬ 
tection  system  which  endeavors  to  provide  a  reasonable  estimate  of  signal 
present.  Herein,  the  integral  of  the  signal-noise  quantity  Y  is  seen  to 
represent  a  mapping  of  a  many-membered  set  of  time  events  (occurring 
during  the  specified  time  interval)  into  a  single  member  of  another  set, 
where  a  member  of  the  latter  is  not  definable  by  a  unique  combination  of 
members  of  the  initial  set.  This  mapping  of  an  arbitrary  time  set  of  mem¬ 
bers  into  a  single  member  is  a  dictate  of  the  mathematics;  it  may  be  real¬ 
ized  directly  with  physical  equipment  because  Y  happens  to  be  the  quantity 
available  and  integration  something  readily  provided  by  the  hardware. 

Now,  when  we  move  to  the  more  general  case  given  in  section  2.  1  by 
equation  (2-9),  we  see  the  mathematics  requests  a  similar  mapping,  in  this 
case  the  initial  set  of  members  each  representing  a  subset  of  XY  time 
events.  This  mapping  may  not  be  realized  directly,  however,  since  the  XY 
Subset  is  unavailable.  It  requires  explicit  knowledge  of  X:  the  very  thing 
we  seek.  Thus,  it  appears  that  the  exact  solution  is  not  directly  attainable. 

For  argument  sake,  let  us  set  exactness  aside  for  the  moment,  and  say  a 
solution  which  is  "reasonably  exact"  will  be  permitted.  Under  this  quali¬ 
fication  of  exactness,  we  may  reason  an  approximate  solution  wherein  the 
approximation  is  predicated  on  a  presumed  mapping  operation  on  the  un¬ 
known  quantity  X.  We  need  to  choose  a  mapping  on  X,  however,  such  that 
the  required  mapping  of  XY  is  realizable,  that  is,  the  integral  of  XY  solv¬ 
able. 
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Suppose  for  the  present  we  consider  a  mapping  of  X,  as  described  by  a 
subset  of  time  events  over  interval  (O,  T),  into  a  single  member  of  another 
set  where  this  member  represents  a  measure  of  X  over  interval  (O,  T). 

The  simplest,  and  most  obvious  such  member  set  is  that  one  whose  mem¬ 
bers  represent  that  measure  of  X  defined  by  the  time -average  of  X  over 
interval  (O,  T).  This  presumed  mapping  of  X  is  irreversible,  with  infor¬ 
mation  lost  in  the  process.  It  would  be  acceptable  as  long  as  a  sub-mem¬ 
ber  of  the  initial  set  does  not  differ  significantly  from  a  chosen  member  in 
the  final  set.  This  is  to  say  that  such  an  assumed  mapping  of  X  is  per¬ 
mitted  if  X  is  reasonably  behaved,  with  reasonable  behavior  determined 
by  some  fidelity  criterion  (as  allowable  variance  for  deviation  of  X  during 
interval  from  the  average  of  X  over  the  interval). 

Let  us  now  translate  this  rather  simple  argument  to  a  general  time  func¬ 
tion  for  X.  Such  a  function  is  shown  in  figure  C-l.  Therein,  we  may  di¬ 
vide  an  arbitrarily  long  period  of  time  into  nT  sub-intervals,  and  assume 
the  function  can  be  approximated  by  a  step -like  function  described  by  a 
constant  value  for  each  sub-interval.  Each  constant  value  may  be  the  time- 
average  of  X  over  the  corresponding  sub-interval.  This  step-like  approxi¬ 
mation  of  X  does  indeed  assume  an  irreversible  mapping  on  X  wherein 
some  knowledge  of  X  is  lost.  As  figure  C-l  indicates,  however,  so  long 
as  the  length  T  of  each  sub-interval  is  chosen  that  a  prescribed  fidelity 
criterion  is  satisfied,  the  step-like  form  is  a  reasonable  representation 
of  X  itself. 

This  being  so,  the  posterior  statement  for  the  constant  signal  case  given 
in  section  2.  1,  equation  (2-10)  is  then  a  fair  statement  for  this  arbitrary 
signal  X  during  any  one  of  the  sub-intervals.  Moreover,  the  integral  of 
the  signal-noise  quantity  Y  reasonably  satisfies  the  sufficiency  conditions 
for  a  detection  system,  that  is,  the  general  integral  XY  is  reasonably 
approximated. 

At  a  glance,  there  appears  to  be  nothing  profound  in  the  foregoing  argu¬ 
ment;  use  of  the  approximate  solution  therefrom  would  not  be  an 


C-2 


innovation  —  it  has  a  long  practiced  history.  Reflecting  a  moment,  how¬ 
ever,  we  see  in  our  thinking  the  seed  for  general  problem  solution;  it  lies 
in  our  previous  statement  regarding  a  presumed  mapping  of  X  which  is  so 
chosen  that  the  eventual  mapping  of  XY  is  realizable,  and  hence  the  inte¬ 
gral  of  XY  solvable. 

When  we  speak  of  a  presumed  mapping  of  X,  we  really  are  only  seeking  a 
more  useful  way  of  representing  X.  We  see  difficulty  in  forming  the  inte¬ 
gral  XY  directly  only  because  X  is  seen  as  an  infinite  set  of  unknown  time 
samples.  In  a  way,  the  task  would  seem  less  difficult  if  an  arbitrary  X 
Were  represented  by  a  finite  set  of  samples  through  use  of  the  sampling 
theorem;  formation  of  the  required  integrals  involve  sin  wt/t  type  func¬ 
tions,  however,  not  permitting  direct  implementation  of  this  representation. 

With  an  eye  always  on  the  practicality  of  implementing  the  required  inte¬ 
gration  resulting  from  any  new  X  representation,  we  might  then  consider 
the  signal  representation  used  in  improving  serial,  general-purpose  digi¬ 
tal  computer  operation.  Therein,  a  power  series  in  time  was  employed  as 
the  general  form  to  represent  a  signal,  because  of  its  ease  in  implementa¬ 
tion. 

If  X  is  represented  by  a  power  series  in  time  with  arbitrary  coefficients, 
the  solution  of  the  XY  integral  is  direct.  Its  implementation,  moreover, 
is  straightforward,  involving  only  successive  integrals  of  Y.  The  arbi¬ 
trary  coefficients  need  not  be  prestated;  they  may  be  chosen  at  a  later  date. 

Nor  are  we  limited  to  power  series  alone;  we  may  employ  the  Fourier 
series,  for  example.  We  see  then  the  solution  of  the  XY  integral  to  call 
for  an  infinite  set  of  filters  matched  to  the  sinusoid  of  each  harmonic. 

The  arbitrary  coefficients  of  the  Fourier  series  can  again  be  chosen  at  a 
later  date. 
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VARIANCE  OF  A  TIME -AVERAGE  QUANTITY 


Suppose  we  request  the  variance  of  time-average  of  some  quantity  X  as 
given  by 
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X(i>At)At 
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wherein  it  is  understood  that  the  summation  expression  holds  as  At 
approaches  zero. 


Utilizing  the  infinite  sum  in  equation  (D-l)  and  also  making  use  of  the  cor¬ 
relation  coefficient  of  X,  that  is  Px(t)  where  t  represents  the  time  sepa¬ 
ration  between  any  two  X  values,  the  desired  variance  can  be  written  in 
infinite  sum  form  as 

aJt  =  |y  +  0*  +  cr 2Zt  +  •  •  •  +  cr(J.i)AtJ 
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where  CQ  ,  <7^,  CT^  *  '  '»  c(u_i)^t  are  variances  of  X  at  times  o, 

At,  2At,  •  •  •,  (u-l)At.  The  quantity  v  is  equal  to  T/At.  Advantage  was 
taken  in  the  symmetry  of  the  correlation  coefficient. 


Since  the  sum  involves  the  same  variable,  then  all  variances  appearing 
in  equation  (D-2)  are  equal,  that  is. 


Employing  this  fact  reduces  equation  (D-2)  to 
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which  may  be  written  as 
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and  further  as 


+■<& 


?  T_  1 

?  2a  4"St  u  , 

e  +  —X  v  V  P|nAt)At‘2 
T  7^1  h^l 


(D-6) 


D-2 


In  the  limit  as  At  approaches  zero,  the  first  term  on  the  right-hand  side 
of  equation  (D-6)  goes  to  zero  and  the  second  term  can  be  replaced  by  a 
double  integral,  that  is,  the  desired  variance  becomes 


(D-7) 


As  an  aside,  Lee  Ogives  an  equivalent  form  of  equation  (D-7)  as 


P(t)dt 
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(D-8) 


wherein  his  notation  has  been  altered  to  conform  with  ours.  The  equiv¬ 
alence  is  seen  by  rewriting  equation  (D-8)  as 
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Employing  integration  by  parts  on  the  second  term,  we  have 


(D- 10) 


Substituting  equation  (D-10)  into  equation  (D-9)  shows  the  equivalence 


(D-ll) 


*  Reference  8  (Lee),  p.  286. 
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APPENDIX  E 


ALTERNATE  DERIVATION  OF  TIME -AVERAGE  ESTIMATE 
FOR  BAND -LIMITED  WHITE  GAUSSIAN  NOISE 


Of  some  interest  is  an  alternate  derivation  of  the  approximate  posterior 
statement  for  signal  detection  in  presence  of  band-limited,  white  Gaussian 
noise  with  zero  mean  in  terms  of  the  time -average  estimates.  Reference 
here  is  to  the  approximate  posterior  statement  given  in  section  4.  2, 
equation  (4-14).  This  equation  is  repeated  below: 
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With  noise  having  zero  mean,  equals  zero.-  Only  a— t  remains  to  be 

determined.  This  variance  quantity  was  defined  in  equation  (4-15)  of 
section  4.  2  as 
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The  correlation  coefficient  p^(t)  in  equation  (E-2)  is  obtainable  by  nor¬ 
malizing  the  autocovariance  for  the  noise  quantity,  that  is, 

tow 
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where  4^(w)  is  the  noise  power  density  spectrum  given  by 
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Thus,  equation  (E-3)  reduces  to 


%(t)  =  w  =  ^  ["  (E-5) 


*'  Discussion  of  band-limited,  white  Gaussian  noise  may  be  found  in 
Reference  9  (Bennett). 
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Upon  substituting  equation  (E-5)  into  equation  (E-2),  we  have 
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The  results  of  equation  (E-6)  point  to  some  inconsistency  in  assuming  a 
finite  bandwidth  and  independent  noise  samples.  This  inconsistency  is 
considered  more  of  academic  interest,  however,  brought  about  by  the 
noise  model  chosen.  Normal  interpretation  of  equation  (E-6)  would  con¬ 
sider  the  case  where  bandwidth  approaches  infinity  (analogous  to  zero 
correlation  between  samples),  that  is, 

Lim  V  sin2ffWt  JT  _ 

W—  /  - 1 -  dt  "  2  (E’7 
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or  at  least  a  very  large  W  where  equation  (E-7)  holds  nearly  always. 
Inserting  this  result  into  equation  (E-6)  provides 
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Utilizing  this  quantity  in  equation  (E-l)  gives  the  approximate  posterior 
statement  as 

t  2  “t  X 

P(X/Y)  =  KP(X‘)  exp{-  1^}  exp  {^-  J  Ydt}  (E-9) 

which  is  in  exact  agreement  with  the  previously  determined  result  given 
in  section  4.  1  by  equation  (4-13). 
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APPENDIX  F 


ERROR  IN  STEP-LIKE  REPRESENTATION 
OF  GENERAL  TIME  FUNCTION 


Let  us  determine  the  probable  error  in  representing  a  general  time  func¬ 
tion  X(t)  in  step-like  form.  This  representation  is  sketched  in  figure  F-l 
As  shown,  X(t)  is  approximated  by  a  constant  value  for  each  T-second 
time  interval.  More  specifically,  these  constant  values  are  deemed  to 
constitute  the  time-averages  of  X(t)  for  the  corresponding  time  intervals. 

We  now  inquire  into  the  error  resulting  from  such  approximations.  To 
satisfy  our  curiosity  we  need  only  concentrate  on  one  arbitrary  T-second 
interval.  Such  an  interval  is  illustrated  in  figure  F-2.  For  convenience, 
the  time  base  thereon  is  now  denoted  by  a  starting  time  t  equal  to  zero 
coinciding  with  the  beginning  of  the  chosen  T  interval. 


The  general  error  expression  for  the  approximation  during  the  arbitrary 
time  interval  of  figure  F-2  may  be  written  as 

«  -t  =  X(t)  -  X1 
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Let  us  assume  this  error  to  be  sufficiently  characterized  if  we  ascertain 
its  mean  value  and  its  variance.  Accepting  this,  we  first  rewrite 
equation  (F-l)  to  represent  the  integral  by  an  infinite  sum.  We  may  do 
this  by  subdividing  the  interval  into  T/At  equal  time  increments,  giving 
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where  it  is  understood  that  this  equation  is  valid  in  the  limit  as  At  ap¬ 
proaches  zero,  and  that  t  ranges  between  zero  and  T. 


The  form  of  equation  (F-2)  now  readily  lends  itself  to  a  determination  of 
the  mean  and  variance  of  the  error. 


Since  the  mean  of  a  sum  or  difference  is  equal  to  the  sum  or  difference 
of  the  individual  means,  the  error  mean  may  be  written  as 
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where  /it  and  juq,  M2^t>  ‘  *  *>  •‘(x-ljAt  are  the  mean  values  of  X(t) 

at  times  t  and  o,  At,  ZAt,  •  •  •  ,  (v-l)At  respectively.  Assuming  a  sta¬ 
tionary  stochastic  process,  all  means  appearing  in  equation  (F-3)  are  the 
same,  being  the  mean  of  X(t).  Denoting  this  mean  by  fi  allows  reduction 
of  equation  (F-3)  to 
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Proceeding  in  a  like  manner  we  may  write  the  expression  for  the  variance 
of  equation  (F-2)  as 
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wherein  advantage  is  taken  of  the  fact  that  the  correlation  coefficient 
Px(t)  is  an  even  function  in  t  (symmetric  about  t  equal  zero).  Now  all  cr 
values  appearing  in  equation  (F-5.)  are  the  same,  where  the  common 
value  will  be  denoted  by  a .  Hence,  equation  (F-5)  can  be  written  in 
more  compact  form  as 
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which  can  be  further  compressed  as 


T-t-At 


_  2  .  2  /aA  2  2cTx  t/At 

a  «  -  ax  + -  T 


2a  At 

2  Px(u At )At  -  Px(uAt)At 

v -o  v  =1 


2„2sf-\  , 

+-$£  E^41'41 


(F-7) 


u=l  r)  =o 

Allowing  At  in  equation  (F-7)  to  approach  zero  in  the  limit  permits  the 
sums  to  be  replaced  by  integrals,  and  causes  the  second  term  to  go  to 
zero,  that  is, 
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Since  the  variance  of  equation  (F-8)  varies  over  the  interval,  it  is  of  in¬ 
terest  to  determine  its  minimum  and  maximum  values.  Note  Px(t),  as 
utilized  in  the  derivation,  is  always  zero  or  positive  valued,  that  is, 

0  *  Px(t)  ^  l  (F-9) 


Hence,  all  integrals  appearing  in  equation  (F-8)  are  positive,  monotoni- 
cally  increasing  functions.  Moreover,  from  the  structure  of  this  equation 
we  see  the  minimum  will  occur  when  the  sum  of  the  first  two  integrals  is 
,  a  maximum.  Since  in  practical  situations  Px(t)  itself  will  be  a  monotoni- 
cally  decreasing  function  over  prescribed  interval  of  length  T,  this  sum 
has  its  maximum  when  their  upper  limits  are  both  equal  to  T/2,  or  the 
mid-point  of  the  interval.  Thus,  the  minimum  variance  of  the  error  be¬ 
comes 

Through  similar  reasoning,  it  can  be  shown  that  the  maximum  variance 
occurs  at  the  end-points  of  the  interval,  having  a  value 
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The  conclusions  of  equation  (F-10)  and  (F-ll)  are  in  agreement  with  our 
intuitive  notions.  We  would  expect  the  error  to  be  least  at  the  mid-point 
and  largest  at  the  end-points. 


As  a  final  item,  let  us  restate  equation  (F-10)  for  a  Taylor  series  repre¬ 
sentation  of  Px(t).  This  form  leads  to  ready  interpretation,  especially 
since  the  behavior  of  Px(t)  over  the  range  of  interest  often  reduces  to  two 
or  three  terms  of  the  Taylor  series.  The  general  Taylor  series  for  P  (t) 
expanded  about  t  equal  zero  is 
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and  where  it  is  understood  the  derivatives  are  determined  by 
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with  t  being  positive.  This  right-hand  interpretation  of  the  derivatives 
circumvents  arguments  which  might  arise  because  derivatives  at  t  equal 
.  zero  may  be  undefined,  since  Px(t)  is  even,  p^(0)  is  a  maximum,  and  a 
point  of  discontinuity  can  occur  at  t  equal  zero.  Actual  concern  with 
equation  (F-15)  is  not  envisioned  in  general  application,  however. 


Utilizing  equation  (F-12)  in  equation  (F-10)  and  integrating  results  in 
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wherein  use  is  made  of  equation  (F-13). 


Since  equation  (F-13)  always  is  true,  the  expression  of  equation  (F-16)  is 
directly  applicable  to  a  Px(t)  comprised  of  a  Taylor  series  representable 
function  plus  a  delta  function.  The  delta  function  would  of  course  indicate 
presence  of  white  noise.  It  only  affects  the  p^  term,  however,  which  we 
see  to  be  a  constant  equal  to  unity:  hence  the  validity  of  equation  (F-16), 


APPENDIX  G 
INFORMATION  GAIN 

As  we  ultimately  desire  to  evaluate  effectiveness  of  detection  operation, 
some  measure  of  information  gain  would  be  most  appropriate.  Certain 
mathematical  difficulties  arise,  however,  with  regard  to  information  gain 
measures  for  continuous  distributions. 

In  general,  some  feeling  for  information  gain  can  be  obtained  by  compar¬ 
ing  some  function  of  the  prior  variance  with  the  same  function  of  the 
posterior  variance.  This  feeling  can  be  mathematically  expressed  through 
the  classical  definition  of  entropy  change,  which  reflects  an  average  meas¬ 
ure  of  information  gain. 

Since  entropy  for  a  continuous  probability  density  distribution  is  defined  by 
00 

H(X)  =  -  /p(X)  log  p(X)  dX  (G-l) 

*00 

This  definition  may  be  used  to  express  average  information  gain  as  the 
entropy  differential  between  the  a  priori  distribution  p(X)  and  the  a 
posterior  distribution  p(X/Y),  that  is, 

AH  =  H(X)  -  H(X/y) 

00  00 

=  -  f  p(X)  log  p(X)dX  +  f  p(X/Y)  log  p(X/  Y)dX  (G-2) 

-00  *^.00 

Reza*  refers  to  this  entropy  change  as  "transinformation". 

To  reflect  some  of  the  unattractive  behavior  of  equation  (G-2),  let  us 

2 

examine  the  case  where  the  distributions  are  Gaussian.  Assuming  <x 

2  x 
to  be  the  a  priori  variance  and  y  to  b®  the  a  posteriori  variance, 

equation  (G-2)  becomes 


*  Reference  10  (Reza). 


G-l 


(G-3) 


The  normal  argument  associated  with  equation  (G-3)  states  that  one  half 
the  logarithm  of  the  ratio  of  the  a  priori  to  the  a  posteriori  variance  is  an 
average  measure  of  information  gain.  A  reduction,  therefore,  in  the  pos¬ 
terior  variance  from  the  prior  variance  results  in  a  positive  entropy 
change  denoting  a  gain  in  information.  Generally  the  equation  (G-3)  def¬ 
inition  finds  effective  usage  for  continuous  signal  data,  if  care  is  taken  to 
avoid  some  of  its  mathematical  difficulty. 


One  such  difficulty  arises  when  the  posterior  variance  becomes  zero.  The 
result  is  an  infinite  positive  change  in  entropy,  implying  infinite  informa¬ 
tion  gain.  So  long  as  infinite  entropy  change  is  understood  to  mean  that 
all  information  has  been  obtained,  the  definition  is  satisfactory.  As  a 
numerical  measure  where  the  posterior  variance  approaches  zero,  it  has 
no  quantitative  meaning,  however. 


Actually  the  foregoing  difficulty  reflects  shortcomings  in  extending  the 
definition  of  entropy  for  discrete  distributions  to  the  continuous.  We  might 
say  the  mathematics  breaks  down.  Entropy  for  continuous  distributions 
can  run  the  spectrum  from  minus  infinity  to  plus  infinity,  thereby  losing 
some  of  its  significance  in  measuring  information  content.  In  a  sense, 
equation  (G-3)  is  only  half  as  bad  in  signal  detection,  in  that  entropy 
change  will  only  cover  the  plus  side  of  the  spectrum  (that  is,  so  long  as 
interpretation  of  information  does  not  take  place).  The  plus  infinity  point 
usually  occurs  in  the  limit  when  the  continuous  density  function  becomes 
a  discrete  probability.  #  In  equation  (G-3)  this  occurs  when  the  a  posteriori 
probability  becomes  a  delta  function. 


*  Reference  10  (Reza),  Chapter  8. 


G-2 


Rather  than  the  average  information  gain  definition  of  equation  (G-3),  in¬ 
formation  gain  which  is  not  averaged  can  be  obtained  directly  from  the 
basic  definition  of  information,  that  is,  by 


(0-3) 


or  the  prior  information  minus  the  posterior  information.  Again  as  for 
equation  (G-3),  continuous  distributions  can  cause  this  expression  to  be¬ 
come  infinite. 


To  avoid  the  foregoing,  we  need  an  expression  for  information  gain  whose 
boundaries  are  zero  and  some  direct  measure  of  actual  information  re¬ 
maining  to  be  gained.  If  we  dispense  with  logarithmic  behavior  in  such  an 
expression,  the  straightforward  relation  of  fractional  difference  between 
prior  and  posterior  variance  could  be  used,  that  is, 

where  IC  means  information  change  (that  is,  a  change  in  the  information 
to  be  gained).  Thus,  when  no  information  is  gained,  the  quantity  is  zero; 
when  all  information  is  gained  the  quantity  is  unity.  We  should  refrain 
from  calling  equation  (G-4)  information  gain,  however. 


Though  the  equation  (G-4)  has  its  deficiencies,  it  is  nonetheless  simple 
and  effective  for  comparative  purposes  in  analysis.  At  least  it  is  well 
behaved. 
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APPENDIX  H 


CORRELATION  COEFFICIENT  FOR  SECOND  PROBABILITY 
DISTRIBUTION  INVOLVING  TIME -AVERAGE  OF  SIGNAL 


Let  us  derive  the  expression  for  the  correlation  coefficient  appearing  in 
a  Gaussian  second  probability  distribution  for  the  time -average  of  a  sig¬ 
nal.  In  general  form,  this  correlation  coefficient  is  defined  by 


P-t 

x 


cr — t  cr — t 


n 


n-1 


X 

a-t 

X 


(H-l) 


where  the  subscript  n  relates  to  the  time -average  over  the  interval  and 
n-1  to  the  time -average  over  the  previous  interval,  with  advantage  taken 
of  the  fact  that 


= 


and 


O— t  =  c— t  =  a— t 

x  n  x  n-1  x 


Equation  (H-l)  reduces  to 


(H-2) 


(H-3) 
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a-t 
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ait 
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(H-4) 


Only  the  product  term  remains  to  be  resolved.  Expressing  this  term  as 

,H-s 

and  subdividing  both  T  time  intervals  into  vAt  sub-intervals,  allows  ex¬ 
pression  of  the  integral  terms  as  sums,  that  is, 


=  (t)  [^-T)+x(-T+^t)+*  *  '  +X(-2At)+X(-At)J 
|x{0)+X(At)+X(2At)+*  *  *  +  X((v-l)Atj 


(H-6) 


To  form  the  mean  of  the  above  product  then  involves  sums  of  product 
terms  of  the  general  form  expressed  below  for  the  ith  sample:  in  one  term 
and  the  jth  sample  in  the  other: 

*(iAt)  X[jAtj  =  U  *+Px((i-j)A  t^ax2=Mx2+Px((j-  i)At^  ax2  (H-7) 

? 

where  a“  is  the  variance  of  X  and  p  (t)  its  correlation  coefficient, 
x  x 

2  2 

Noticing  there  are  v  of  the  terms,  the  mean  of  equation  (H-6)  may  be 
written  as 


3  ,] 


+  Px(T+(u-l)At)| 


+  <7 


Px(T-At)+Px(T)+Px(T+At)+  • 


Px(T-2At)+Px(T-At)+Px(T)+  - 


+Px(T+(u-2)At) 


+Px(T+(u-3)At) 
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+  <7_, 


+  O 


which  may  be  stated  as 


Px(2At)+Px(3At)+Px(4At)  + 


Px(Pt)+Px(2Pt)+Px(3At)+ 


•  •  •  +Px(T+At)j 

•  *  +PX<T)]  } 


(H-8) 
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(xn-l)(xl)=  »x+-p.  (At) 
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Y  Px(nAt)At  +  ^  Px(nAt)P,t  +  ' 
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and  in  integral  form  as 


or 


p„(t)dt4 


Upon  substituting  into  equation  (H-4),  we  have 


(H-9) 
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Px<t)dt‘ 
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(H-l  1) 


and  using  the  definition  of  cr-^t  from  Appendix  D,  equation  (H-ll)  becomes 


(H-l  2) 


H-3 


APPENDIX  I 


CORRELATION  COEFFICIENT  FOR  SECOND 
PROBABILITY  DISTRIBUTION  INVOLVING 
TIME-AVERAGE  AND  MID-POINT  OF  SIGNAL 


The  general  expression  for  the  correlation  coefficient  of  the  Gaussian 
second  probability  distribution  involving  time -average  and  mid-point  of 
signal  can  be  written  as 
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where 


and 


=  4-t  =  a 
x  'x 


(1-2) 


*M 
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Equation  (1-1)  reduces  to 
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(1-4) 


To  determine  the  mean  of  the  product  term,  it  is  first  rewritten  as 


—  - 1 

XMXt  =  *m[t  ^Xdt]  =  T  X(l)S  X(,/At)At 
where  At  approaches  zero  and 


(1-5) 


(1-6) 


1-1 


Writing  out  the  sum,  equation  (I- 5)  becomes 

X(0)tX(M)+X(2H|f  '  ’  +X^(»-l>*j| 


T  41 


Recognizing  that 


X(t)X("“I)=  “*  +»x^('“  •  I) 


we  have 


which  may  be  written  in  integral  form  as 
- r  ,  2a  2  Tj2 

V1  pxlt,dt 


Upon  inserting  equation  (I- 10)  into  equation  (1-4),  we  then  have 
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where  the  expression  for  a—t  is  obtained  from  Appendix  D. 
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